{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Capital Bikeshare共享单车骑行量预测——线性回归分析\n",
    "在本次作业中，我们将利用Capital Bikeshare公司的共享单车数据训练和测试相关模型，并对模型的性能和预测能力进行测试。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np  # 矩阵操作\n",
    "import pandas as pd # SQL数据处理\n",
    "\n",
    "from sklearn.metrics import mean_squared_error  #评价回归预测模型的性能\n",
    "\n",
    "import matplotlib.pyplot as plt   #画图\n",
    "import seaborn as sns\n",
    "\n",
    "# 图形出现在Notebook里而不是新窗口\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. 读取数据\n",
    "我们将做完特征工程后的数据读取到本实例中，数据文件为：FE_day.csv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>instant</th>\n",
       "      <th>season_1</th>\n",
       "      <th>season_2</th>\n",
       "      <th>season_3</th>\n",
       "      <th>season_4</th>\n",
       "      <th>mnth_1</th>\n",
       "      <th>mnth_2</th>\n",
       "      <th>mnth_3</th>\n",
       "      <th>mnth_4</th>\n",
       "      <th>mnth_5</th>\n",
       "      <th>...</th>\n",
       "      <th>weekday_5</th>\n",
       "      <th>weekday_6</th>\n",
       "      <th>temp</th>\n",
       "      <th>atemp</th>\n",
       "      <th>hum</th>\n",
       "      <th>windspeed</th>\n",
       "      <th>cnt</th>\n",
       "      <th>holiday</th>\n",
       "      <th>workingday</th>\n",
       "      <th>yr</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.355170</td>\n",
       "      <td>0.373517</td>\n",
       "      <td>0.828620</td>\n",
       "      <td>0.284606</td>\n",
       "      <td>0.110792</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.379232</td>\n",
       "      <td>0.360541</td>\n",
       "      <td>0.715771</td>\n",
       "      <td>0.466215</td>\n",
       "      <td>0.089623</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.171000</td>\n",
       "      <td>0.144830</td>\n",
       "      <td>0.449638</td>\n",
       "      <td>0.465740</td>\n",
       "      <td>0.152669</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.175530</td>\n",
       "      <td>0.174649</td>\n",
       "      <td>0.607131</td>\n",
       "      <td>0.284297</td>\n",
       "      <td>0.177174</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.209120</td>\n",
       "      <td>0.197158</td>\n",
       "      <td>0.449313</td>\n",
       "      <td>0.339143</td>\n",
       "      <td>0.181546</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 35 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   instant  season_1  season_2  season_3  season_4  mnth_1  mnth_2  mnth_3  \\\n",
       "0        1         1         0         0         0       1       0       0   \n",
       "1        2         1         0         0         0       1       0       0   \n",
       "2        3         1         0         0         0       1       0       0   \n",
       "3        4         1         0         0         0       1       0       0   \n",
       "4        5         1         0         0         0       1       0       0   \n",
       "\n",
       "   mnth_4  mnth_5 ...  weekday_5  weekday_6      temp     atemp       hum  \\\n",
       "0       0       0 ...          0          1  0.355170  0.373517  0.828620   \n",
       "1       0       0 ...          0          0  0.379232  0.360541  0.715771   \n",
       "2       0       0 ...          0          0  0.171000  0.144830  0.449638   \n",
       "3       0       0 ...          0          0  0.175530  0.174649  0.607131   \n",
       "4       0       0 ...          0          0  0.209120  0.197158  0.449313   \n",
       "\n",
       "   windspeed       cnt  holiday  workingday  yr  \n",
       "0   0.284606  0.110792        0           0   0  \n",
       "1   0.466215  0.089623        0           0   0  \n",
       "2   0.465740  0.152669        0           1   0  \n",
       "3   0.284297  0.177174        0           1   0  \n",
       "4   0.339143  0.181546        0           1   0  \n",
       "\n",
       "[5 rows x 35 columns]"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv(\"FE_day.csv\")\n",
    "\n",
    "#通过观察前5行，了解数据每列（特征）的概况\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###  数据基本信息\n",
    "样本数目、特征维数\n",
    "每个特征的类型、空值样本的数目、数据类型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 731 entries, 0 to 730\n",
      "Data columns (total 35 columns):\n",
      "instant         731 non-null int64\n",
      "season_1        731 non-null int64\n",
      "season_2        731 non-null int64\n",
      "season_3        731 non-null int64\n",
      "season_4        731 non-null int64\n",
      "mnth_1          731 non-null int64\n",
      "mnth_2          731 non-null int64\n",
      "mnth_3          731 non-null int64\n",
      "mnth_4          731 non-null int64\n",
      "mnth_5          731 non-null int64\n",
      "mnth_6          731 non-null int64\n",
      "mnth_7          731 non-null int64\n",
      "mnth_8          731 non-null int64\n",
      "mnth_9          731 non-null int64\n",
      "mnth_10         731 non-null int64\n",
      "mnth_11         731 non-null int64\n",
      "mnth_12         731 non-null int64\n",
      "weathersit_1    731 non-null int64\n",
      "weathersit_2    731 non-null int64\n",
      "weathersit_3    731 non-null int64\n",
      "weekday_0       731 non-null int64\n",
      "weekday_1       731 non-null int64\n",
      "weekday_2       731 non-null int64\n",
      "weekday_3       731 non-null int64\n",
      "weekday_4       731 non-null int64\n",
      "weekday_5       731 non-null int64\n",
      "weekday_6       731 non-null int64\n",
      "temp            731 non-null float64\n",
      "atemp           731 non-null float64\n",
      "hum             731 non-null float64\n",
      "windspeed       731 non-null float64\n",
      "cnt             731 non-null float64\n",
      "holiday         731 non-null int64\n",
      "workingday      731 non-null int64\n",
      "yr              731 non-null int64\n",
      "dtypes: float64(5), int64(30)\n",
      "memory usage: 200.0 KB\n"
     ]
    }
   ],
   "source": [
    "df.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 数据准备"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 从原始数据中分离输入特征x和输出y\n",
    "y = df[\"cnt\"]\n",
    "\n",
    "X = df.drop([\"cnt\"], axis = 1)\n",
    "\n",
    "#特征名称，用于后续显示权重系数对应的特征\n",
    "feat_names = X.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(584, 34)"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#将数据分割训练数据与测试数据\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "# 随机采样20%的数据构建测试样本，其余作为训练样本\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=33, test_size=0.2)\n",
    "X_train.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3、确定模型类型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.1 尝试训练最小二乘线性回归模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>coef</th>\n",
       "      <th>columns</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>33</th>\n",
       "      <td>0.523551</td>\n",
       "      <td>yr</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>0.305429</td>\n",
       "      <td>temp</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>0.148181</td>\n",
       "      <td>mnth_9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>0.114507</td>\n",
       "      <td>atemp</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>0.106982</td>\n",
       "      <td>mnth_10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>0.105201</td>\n",
       "      <td>weathersit_1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.095557</td>\n",
       "      <td>season_4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>0.067452</td>\n",
       "      <td>mnth_12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>0.059572</td>\n",
       "      <td>mnth_8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>0.047123</td>\n",
       "      <td>weathersit_2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>0.046841</td>\n",
       "      <td>mnth_11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>0.027430</td>\n",
       "      <td>weekday_6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>0.024960</td>\n",
       "      <td>workingday</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>0.021836</td>\n",
       "      <td>mnth_6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.009795</td>\n",
       "      <td>season_2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>0.008918</td>\n",
       "      <td>weekday_5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>0.007647</td>\n",
       "      <td>weekday_3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>0.005022</td>\n",
       "      <td>weekday_4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>0.000576</td>\n",
       "      <td>mnth_5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>-0.000796</td>\n",
       "      <td>instant</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>-0.003675</td>\n",
       "      <td>weekday_2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-0.015049</td>\n",
       "      <td>season_3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>-0.022045</td>\n",
       "      <td>weekday_0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>-0.023297</td>\n",
       "      <td>weekday_1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>-0.023371</td>\n",
       "      <td>mnth_7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>-0.030345</td>\n",
       "      <td>holiday</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>-0.055881</td>\n",
       "      <td>mnth_4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>-0.062292</td>\n",
       "      <td>mnth_3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-0.090303</td>\n",
       "      <td>season_1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>-0.135164</td>\n",
       "      <td>windspeed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>-0.138876</td>\n",
       "      <td>mnth_2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>-0.149401</td>\n",
       "      <td>hum</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>-0.152324</td>\n",
       "      <td>weathersit_3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>-0.171022</td>\n",
       "      <td>mnth_1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        coef       columns\n",
       "33  0.523551            yr\n",
       "27  0.305429          temp\n",
       "13  0.148181        mnth_9\n",
       "28  0.114507         atemp\n",
       "14  0.106982       mnth_10\n",
       "17  0.105201  weathersit_1\n",
       "4   0.095557      season_4\n",
       "16  0.067452       mnth_12\n",
       "12  0.059572        mnth_8\n",
       "18  0.047123  weathersit_2\n",
       "15  0.046841       mnth_11\n",
       "26  0.027430     weekday_6\n",
       "32  0.024960    workingday\n",
       "10  0.021836        mnth_6\n",
       "2   0.009795      season_2\n",
       "25  0.008918     weekday_5\n",
       "23  0.007647     weekday_3\n",
       "24  0.005022     weekday_4\n",
       "9   0.000576        mnth_5\n",
       "0  -0.000796       instant\n",
       "22 -0.003675     weekday_2\n",
       "3  -0.015049      season_3\n",
       "20 -0.022045     weekday_0\n",
       "21 -0.023297     weekday_1\n",
       "11 -0.023371        mnth_7\n",
       "31 -0.030345       holiday\n",
       "8  -0.055881        mnth_4\n",
       "7  -0.062292        mnth_3\n",
       "1  -0.090303      season_1\n",
       "30 -0.135164     windspeed\n",
       "6  -0.138876        mnth_2\n",
       "29 -0.149401           hum\n",
       "19 -0.152324  weathersit_3\n",
       "5  -0.171022        mnth_1"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 线性回归\n",
    "#class sklearn.linear_model.LinearRegression(fit_intercept=True, normalize=False, copy_X=True, n_jobs=1)\n",
    "from sklearn.linear_model import LinearRegression\n",
    "\n",
    "# 1.使用默认配置初始化学习器实例\n",
    "lr = LinearRegression()\n",
    "\n",
    "# 2.用训练数据训练模型参数\n",
    "lr.fit(X_train, y_train)\n",
    "\n",
    "# 3. 用训练好的模型对测试集进行预测\n",
    "y_test_pred_lr = lr.predict(X_test)\n",
    "y_train_pred_lr = lr.predict(X_train)\n",
    "\n",
    "\n",
    "# 看看各特征的权重系数，系数的绝对值大小可视为该特征的重要性\n",
    "fs = pd.DataFrame({ \"coef\":list((lr.coef_.T)),\"columns\":list(feat_names)})\n",
    "fs.sort_values(by=['coef'],ascending=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3.1.1 模型评价"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The RMSE of LinearRegression on test is 0.09370397005135352\n",
      "The RMSE of LinearRegression on train is 0.0854526405057376\n"
     ]
    }
   ],
   "source": [
    "# 使用RMSE:mean_squared_error评价模型在测试集和训练集上的性能，并输出评估结果\n",
    "#测试集\n",
    "import math\n",
    "mset = ( mean_squared_error(y_test, y_test_pred_lr))\n",
    "print ('The RMSE of LinearRegression on test is',math.sqrt(mset))\n",
    "#训练集\n",
    "msea= (mean_squared_error(y_train, y_train_pred_lr))\n",
    "print ('The RMSE of LinearRegression on train is',math.sqrt(msea))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The r2 score of LinearRegression on test is 0.8279474225980328\n",
      "The r2 score of LinearRegression on train is 0.8516480637403496\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import r2_score \n",
    "# 使用r2_score评价模型在测试集和训练集上的性能，并输出评估结果\n",
    "#测试集\n",
    "print ('The r2 score of LinearRegression on test is', r2_score(y_test, y_test_pred_lr))\n",
    "#训练集\n",
    "print ('The r2 score of LinearRegression on train is', r2_score(y_train, y_train_pred_lr))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAekAAAFsCAYAAADos0H8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAHsBJREFUeJzt3X+cXXV95/HXh2QglphAIImBqAkaUH4lwQFJKRAbEdogYR+rouVHWCJY0K7V1Qpa65TqVgvV6kJ9mPUHQbAEsqVhde1CUyjCKjTBFIGgiUgkISZDlB8BERI++8c9E4dkJnNn5s7Md+a+no/HPO49537vOZ/vvcm87/ecM98bmYkkSSrPXkNdgCRJ6pohLUlSoQxpSZIKZUhLklQoQ1qSpEIZ0pIkFcqQVtOLiAcjYu5Q1zGUIuI/RcRjEbEtImYP4n63RcQh3Tx2fkTc1aD9PBoRb23EtqTBZEhrROvql/Ouv/wz84jMvKOH7UyLiIyI0QNU6lC7EvhAZo7NzB/u+mDV92erUN0YEZ+PiFH93Wm1v0f6ux1ppDKkpQIUEP6vBR7soc3MzBwLnAycBVww4FVJTc6QVtPrPNqOiOMiYmVEPB0RmyPi81WzO6vbJ6vR5JyI2Csi/jwi1kfEloi4NiLGd9ruedVjWyPik7vspy0ilkXEdRHxNHB+te/vR8STEbEpIq6KiL07bS8j4pKIWBsRz0TEX0XE66rnPB0RN3Zuv0sfu6w1IvaJiG3AKOA/IuKnPb1embkOuBuY1Wn74yPia1XdGyPi0x0j7Yh4fUT8W0Q8FRFPRMTSXfr0+ur+ARFxS9WXe4HXdWq325GMiLgjIt5b3X9dRPxr9Vo/ERHXR8R+3bwW3b3HUnEMaenlvgh8MTPHUQuJG6v1J1W3+1WHaL8PnF/9vAU4BBgLXAUQEYcDfw+cDUwBxgMH77KvBcAyYD/gemAH8CHgQGAOMA+4ZJfnnAa8CTge+DNgcbWPVwNHAu/ppl9d1pqZv6lGx1AbKb+u66f/VkS8ATgRWNdp9RJgO/B6YDbwNuC91WN/BdwK7A9MBf5HN5u+Gnie2ut1Ab0bqQfw18BBwBupvR5t3bTt7j2WimNIqxn8UzU6fTIinqQWnt15EXh9RByYmdsy8wd7aHs28PnMfCQztwGXAe+uRnvvAP53Zt6VmS8AfwHsOlH+9zPznzLzpcz8dWauyswfZOb2zHwU+Aq1Q8udfS4zn87MB4EHgFur/T8FfJdaQPa21nrdFxHPAmuAO6hex4iYDPwB8KeZ+WxmbgG+ALy7et6L1A6nH5SZz2fmbheDVaPu/wz8RbWNB6gFf10yc11m3lZ96GgHPs/ur12H3rzH0pAypNUMzszM/Tp+2H102tki4FDg4Yj494g4fQ9tDwLWd1peD4wGJlePPdbxQGY+B2zd5fmPdV6IiEMj4tsR8YvqEPh/pzaq7mxzp/u/7mJ5LF3bU631Oqba/lnAm4F9q/WvBVqATZ0+CH0FmFQ9/mfURrr3Ru1K+q5GyBOrejq/Juu7aNeliJgUETdUh9qfBq5j99euQ2/eY2lIGdJSJ5m5NjPfQy1gPgcsi4h92X0UDPA4tYDq8Bpqh3w3A5uoHdoFICJeARyw6+52Wf4y8DAwozoU+3Fq4dYIe6q1bllzI/B9akcHoBasvwEO7PRhaFxmHlE95xeZeWFmHgS8D/j7jvPQnbRX9bx6lxo7PFvd/k6nda/qdP+vqb2eR1ev3Tl089rt4T2WimNIS51ExDkRMTEzXwKerFbvoBYiL1E7n9vhH4APRcT0iBhLbeS7NDO3UzvX/PaI+N3qYq6/pOfAfSXwNLCtOu97ccM6tuda++KzwEUR8arM3ETtnPPfRsS46iK110XEyQAR8c6I6PjA8itqYbqj88Yycwfwj0BbRPxOdU5/YafH24GNwDkRMaoajXc+f/5KYBu1C/sOBj7aXeF7eI+l4hjS0sudBjxYXfH8ReDd1XnU54DPAHdXh3SPB74OfJPald8/o3bR058AVOeM/wS4gdqo+hlgC7URZ3c+AvxR1fZ/Akv30La3uq21LzLzR8C/8dswPA/YG3iIWhAvo3YBGMCxwD3Va3oL8MHM/FkXm/0AtcPpvwCuAb6xy+MXVvvbChwB/L9Oj/0ltcPxTwHfoRb43enyPd5zj6WhEZldHcWT1EjV6PVJaoeyuwooSdqNI2lpgETE26tDt/tSm9HrR8CjQ1uVpOHEkJYGzgJqF2w9DsygdljVQ1eS6ubhbkmSCuVIWpKkQg3qpP4HHnhgTps2bTB3KUlSUVatWvVEZk6sp+2ghvS0adNYuXLlYO5SkqSiRETds+l5uFuSpEL1GNIRcVhErO7083RE/GlETIiI26qvzbstIvYfjIIlSWoWPYZ0Zv44M2dl5ixqX5H3HHAzcCmwIjNnACuqZUmS1CC9PSc9D/hpZq6PiAXA3Gr9EmpfXfexxpUmSc3jxRdfZMOGDTz/vDOUjhRjxoxh6tSptLS09HkbvQ3pd1ObqB9gcjWxPpm5KSImdf80SdKebNiwgVe+8pVMmzaNiEZ9+ZmGSmaydetWNmzYwPTp0/u8nbovHKu+yecM4Kbe7CAiLoqIlRGxsr29vbf1SVJTeP755znggAMM6BEiIjjggAP6fWSkN1d3/wFwX2Z2fP/s5oiYUhUzhdo3/OwmMxdnZmtmtk6cWNefhUlSUzKgR5ZGvJ+9Cen38NtD3VD7yrmO73tdCCzvdzWSJGmnus5JR8TvAKcA7+u0+rPAjRGxCPg58M7GlydJzamtbfC3N2rUKI466ii2b9/O9OnT+eY3v8l+++3X6329973v5cMf/jCHH374y9Zfc801rFy5kquuuqrX2wQYO3Ys27Ztq6vt3LlzufLKK2ltbd25buXKlVx77bV86Utf6tP+h0JdI+nMfC4zD8jMpzqt25qZ8zJzRnX7y4ErU5I00F7xilewevVqHnjgASZMmMDVV1/dp+189atf3S2gS9Da2jrgAb1jx46Gbs8ZxyRJu5kzZw4bN27cuXzFFVdw7LHHcvTRR/OpT30KgGeffZb58+czc+ZMjjzySJYuXQrURrEdU0B/4xvf4NBDD+Xkk0/m7rvv3rm9888/n2XLlu1cHjt2LADbtm1j3rx5HHPMMRx11FEsX777mdRNmzZx0kknMWvWLI488ki+973v1dWnO+64g9NPPx2AtrY2LrjgAubOncshhxzysvC+7rrrOO6445g1axbve9/7dgbvxRdfTGtrK0ccccTO1wBqU15ffvnl/N7v/R433dSra6t7NKhzd0uSyrdjxw5WrFjBokWLALj11ltZu3Yt9957L5nJGWecwZ133kl7ezsHHXQQ3/nOdwB46qmnXradTZs28alPfYpVq1Yxfvx43vKWtzB79uw97nvMmDHcfPPNjBs3jieeeILjjz+eM84442UXYX3rW9/i1FNP5ROf+AQ7duzgueee61M/H374YW6//XaeeeYZDjvsMC6++GLWrVvH0qVLufvuu2lpaeGSSy7h+uuv57zzzuMzn/kMEyZMYMeOHcybN4/777+fo48+emfdd911V5/q2BNDWpIEwK9//WtmzZrFo48+ypve9CZOOeUUoBbSt956686A3bZtG2vXruXEE0/kIx/5CB/72Mc4/fTTOfHEE1+2vXvuuYe5c+fS8Zc9Z511Fj/5yU/2WENm8vGPf5w777yTvfbai40bN7J582Ze9apX7Wxz7LHHcsEFF/Diiy9y5plnMmvWrD71d/78+eyzzz7ss88+TJo0ic2bN7NixQpWrVrFscceu/M1mTSpNg3IjTfeyOLFi9m+fTubNm3ioYce2hnSZ511Vp9q6ImHuyVJwG/PSa9fv54XXnhh5znpzOSyyy5j9erVrF69mnXr1rFo0SIOPfRQVq1axVFHHcVll13G5Zdfvts2u/szpNGjR/PSSy/t3P4LL7wAwPXXX097ezurVq1i9erVTJ48ebe/NT7ppJO48847Ofjggzn33HO59tpr+9TfffbZZ+f9UaNGsX37djKThQsX7uzrj3/8Y9ra2vjZz37GlVdeyYoVK7j//vuZP3/+y+rad999+1RDTxxJS+pSvVcXN/oqZA298ePH86UvfYkFCxZw8cUXc+qpp/LJT36Ss88+m7Fjx7Jx40ZaWlrYvn07EyZM4JxzzmHs2LFcc801L9vOm9/8Zj74wQ+ydetWxo0bx0033cTMmTOB2nncVatW8a53vYvly5fz4osvArVD5pMmTaKlpYXbb7+d9et3/1bH9evXc/DBB3PhhRfy7LPPct9993Heeec1pO/z5s1jwYIFfOhDH2LSpEn88pe/5JlnnuHpp59m3333Zfz48WzevJnvfve7zJ07tyH73BNDWpIKNNQffmbPns3MmTO54YYbOPfcc1mzZg1z5swBahd5XXfddaxbt46PfvSj7LXXXrS0tPDlL3/5ZduYMmUKbW1tzJkzhylTpnDMMcfsvAjrwgsvZMGCBRx33HHMmzdv50j07LPP5u1vfzutra3MmjWLN7zhDbvVdscdd3DFFVfQ0tLC2LFjux1Jz58/f+e82XPmzOH9739/j/0+/PDD+fSnP83b3vY2XnrpJVpaWrj66qs5/vjjmT17NkcccQSHHHIIJ5xwQv0vZj9EZg7KjgBaW1uz44o/SWVzJD241qxZwxvf+MahLkMN1tX7GhGrMrO1m6e8jOekJUkqlCEtSVKhDGlJKsRgnn7UwGvE+2lIS1IBxowZw9atWw3qEaLj+6THjBnTr+14dbckFWDq1Kls2LCB9vb2oS5FDTJmzBimTp3ar20Y0pJUgJaWFqZPnz7UZagwHu6WJKlQhrQkSYUypCVJKpQhLUlSoQxpSZIKZUhLklQoQ1qSpEIZ0pIkFcqQliSpUIa0JEmFMqQlSSqUIS1JUqEMaUmSCmVIS5JUKENakqRCGdKSJBXKkJYkqVCGtCRJhTKkJUkqlCEtSVKhDGlJkgplSEuSVChDWpKkQtUV0hGxX0Qsi4iHI2JNRMyJiAkRcVtErK1u9x/oYiVJaib1jqS/CPxzZr4BmAmsAS4FVmTmDGBFtSxJkhqkx5COiHHAScDXADLzhcx8ElgALKmaLQHOHKgiJUlqRvWMpA8B2oFvRMQPI+KrEbEvMDkzNwFUt5O6enJEXBQRKyNiZXt7e8MKlyRppKsnpEcDxwBfzszZwLP04tB2Zi7OzNbMbJ04cWIfy5QkqfnUE9IbgA2ZeU+1vIxaaG+OiCkA1e2WgSlRkqTm1GNIZ+YvgMci4rBq1TzgIeAWYGG1biGwfEAqlCSpSY2us92fANdHxN7AI8B/oRbwN0bEIuDnwDsHpkRJkppTXSGdmauB1i4emtfYciRJUgdnHJMkqVCGtCRJhTKkJUkqlCEtSVKhDGlJkgplSEuSVChDWpKkQtU7mYmkEaKtbWi21+j9Ss3AkbQkSYUypCVJKpQhLUlSoQxpSZIKZUhLklQoQ1qSpEIZ0pIkFcqQliSpUIa0JEmFMqQlSSqUIS1JUqEMaUmSCmVIS5JUKENakqRCGdKSJBXKkJYkqVCGtCRJhTKkJUkqlCEtSVKhDGlJkgplSEuSVChDWpKkQo0e6gIkNYe2toFpK41kjqQlSSqUIS1JUqEMaUmSCuU5aUnFqfectOeuNdLVFdIR8SjwDLAD2J6ZrRExAVgKTAMeBd6Vmb8amDIlSWo+vTnc/ZbMnJWZrdXypcCKzJwBrKiWJUlSg/TnnPQCYEl1fwlwZv/LkSRJHeoN6QRujYhVEXFRtW5yZm4CqG4nDUSBkiQ1q3ovHDshMx+PiEnAbRHxcL07qEL9IoDXvOY1fShRkqTmVNdIOjMfr263ADcDxwGbI2IKQHW7pZvnLs7M1sxsnThxYmOqliSpCfQY0hGxb0S8suM+8DbgAeAWYGHVbCGwfKCKlCSpGdVzuHsycHNEdLT/Vmb+c0T8O3BjRCwCfg68c+DKlCSp+fQY0pn5CDCzi/VbgXkDUZQkSXJaUEmSimVIS5JUKENakqRCGdKSJBXKkJYkqVCGtCRJhTKkJUkqlCEtSVKh6v2CDUlDoK2tse0kDS+OpCVJKpQhLUlSoQxpSZIKZUhLklQoQ1qSpEIZ0pIkFcqQliSpUIa0JEmFMqQlSSqUIS1JUqEMaUmSCmVIS5JUKENakqRCGdKSJBXKkJYkqVCGtCRJhTKkJUkqlCEtSVKhDGlJkgplSEuSVChDWpKkQhnSkiQVypCWJKlQhrQkSYUypCVJKpQhLUlSoUYPdQGS+q+tbagrGBr19rtZXx8Nf3WPpCNiVET8MCK+XS1Pj4h7ImJtRCyNiL0HrkxJkppPbw53fxBY02n5c8AXMnMG8CtgUSMLkySp2dUV0hExFZgPfLVaDuD3gWVVkyXAmQNRoCRJzarekfTfAX8GvFQtHwA8mZnbq+UNwMFdPTEiLoqIlRGxsr29vV/FSpLUTHoM6Yg4HdiSmas6r+6iaXb1/MxcnJmtmdk6ceLEPpYpSVLzqefq7hOAMyLiD4ExwDhqI+v9ImJ0NZqeCjw+cGVKktR8ehxJZ+ZlmTk1M6cB7wb+NTPPBm4H3lE1WwgsH7AqJUlqQv2ZzORjwIcjYh21c9Rfa0xJkiQJejmZSWbeAdxR3X8EOK7xJUmSJHBaUEmSimVIS5JUKENakqRCGdKSJBXKkJYkqVCGtCRJhTKkJUkqlCEtSVKhDGlJkgplSEuSVChDWpKkQhnSkiQVypCWJKlQhrQkSYUypCVJKpQhLUlSoQxpSZIKZUhLklQoQ1qSpEIZ0pIkFcqQliSpUIa0JEmFMqQlSSqUIS1JUqEMaUmSCmVIS5JUKENakqRCGdKSJBXKkJYkqVCGtCRJhTKkJUkqlCEtSVKhDGlJkgplSEuSVChDWpKkQvUY0hExJiLujYj/iIgHI+Ivq/XTI+KeiFgbEUsjYu+BL1eSpOZRz0j6N8DvZ+ZMYBZwWkQcD3wO+EJmzgB+BSwauDIlSWo+PYZ01myrFluqnwR+H1hWrV8CnDkgFUqS1KTqOicdEaMiYjWwBbgN+CnwZGZur5psAA7u5rkXRcTKiFjZ3t7eiJolSWoKdYV0Zu7IzFnAVOA44I1dNevmuYszszUzWydOnNj3SiVJajK9uro7M58E7gCOB/aLiNHVQ1OBxxtbmiRJza2eq7snRsR+1f1XAG8F1gC3A++omi0Elg9UkZIkNaPRPTdhCrAkIkZRC/UbM/PbEfEQcENEfBr4IfC1AaxTkqSm02NIZ+b9wOwu1j9C7fy0JEkaAM44JklSoQxpSZIKZUhLklQoQ1qSpEIZ0pIkFcqQliSpUIa0JEmFMqQlSSqUIS1JUqEMaUmSCmVIS5JUKENakqRCGdKSJBXKkJYkqVCGtCRJhTKkJUkqlCEtSVKhRg91AVIzamsb6gokDQeOpCVJKpQhLUlSoQxpSZIKZUhLklQoQ1qSpEIZ0pIkFcqQliSpUIa0JEmFMqQlSSqUIS1JUqEMaUmSCmVIS5JUKENakqRCGdKSJBXKkJYkqVCGtCRJhTKkJUkqlCEtSVKhegzpiHh1RNweEWsi4sGI+GC1fkJE3BYRa6vb/Qe+XEmSmkc9I+ntwH/LzDcCxwPvj4jDgUuBFZk5A1hRLUuSpAbpMaQzc1Nm3lfdfwZYAxwMLACWVM2WAGcOVJGSJDWjXp2TjohpwGzgHmByZm6CWpADk7p5zkURsTIiVra3t/evWkmSmkjdIR0RY4H/BfxpZj5d7/Myc3FmtmZm68SJE/tSoyRJTamukI6IFmoBfX1m/mO1enNETKkenwJsGZgSJUlqTvVc3R3A14A1mfn5Tg/dAiys7i8Elje+PEmSmtfoOtqcAJwL/CgiVlfrPg58FrgxIhYBPwfeOTAlSpLUnHoM6cy8C4huHp7X2HIkSVIHZxyTJKlQhrQkSYUypCVJKlQ9F45JUlNoaxuYtlJfOZKWJKlQhrQkSYUypCVJKpTnpNW06j2n6LlHSUPFkbQkSYUypCVJKpQhLUlSoQxpSZIKZUhLklQoQ1qSpEIZ0pIkFcqQliSpUE5mIjWQE59IaiRH0pIkFcqQliSpUIa0JEmFMqQlSSqUF45pxPHiLUkjhSNpSZIKZUhLklQoQ1qSpEJ5Tlrqgee4JQ0VR9KSJBXKkJYkqVCGtCRJhfKctKQRz+sKNFw5kpYkqVCGtCRJhTKkJUkqlCEtSVKhDGlJkgrVY0hHxNcjYktEPNBp3YSIuC0i1la3+w9smZIkNZ96RtLXAKftsu5SYEVmzgBWVMuSJKmBegzpzLwT+OUuqxcAS6r7S4AzG1yXJElNr6/npCdn5iaA6nZSdw0j4qKIWBkRK9vb2/u4O0mSms+AXziWmYszszUzWydOnDjQu5MkacToa0hvjogpANXtlsaVJEmSoO8hfQuwsLq/EFjemHIkSVKHev4E6x+A7wOHRcSGiFgEfBY4JSLWAqdUy5IkqYF6/BaszHxPNw/Na3AtkiSpE2cckySpUIa0JEmFMqQlSSpUj+ekJUm7a2sre3saGRxJS5JUKENakqRCGdKSJBXKkJYkqVCGtCRJhTKkJUkqlCEtSVKhDGlJkgrlZCYaUr2ZwMHJHiQ1G0fSkiQVypCWJKlQhrQkSYUypCVJKpQXjmlAeJGXJPWfI2lJkgplSEuSVChDWpKkQnlOWsOG57k1kjmxj7riSFqSpEIZ0pIkFcqQliSpUJ6TVq94LkwaevX+P/T/6/DnSFqSpEIZ0pIkFcqQliSpUIa0JEmF8sKxPhqICzeG6iIPLy6RpDI5kpYkqVCGtCRJhTKkJUkqVGTmoO2stbU1V65c2bDtDeX5Xs/jSlL3hsPvyKGaFCYiVmVmaz1tHUlLklSofoV0RJwWET+OiHURcWmjipIkSf0I6YgYBVwN/AFwOPCeiDi8UYVJktTs+jOSPg5Yl5mPZOYLwA3AgsaUJUmS+nzhWES8AzgtM99bLZ8LvDkzP7BLu4uAi6rFw4Af17H5A4En+lTY8DDS+wf2caSwj8PfSO8fDL8+vjYzJ9bTsD8zjkUX63ZL/MxcDCzu1YYjVtZ75dtwNNL7B/ZxpLCPw99I7x+M7D7253D3BuDVnZanAo/3rxxJktShPyH978CMiJgeEXsD7wZuaUxZkiSpz4e7M3N7RHwA+L/AKODrmflgg+rq1eHxYWik9w/s40hhH4e/kd4/GMF9HNQZxyRJUv2ccUySpEIZ0pIkFaqIkI6ICRFxW0SsrW7330PbcRGxMSKuGswa+6Oe/kXEayNiVUSsjogHI+KPh6LWvqqzj7Mi4vtV/+6PiLOGota+qvffaUT8c0Q8GRHfHuwa+6qnKX4jYp+IWFo9fk9ETBv8Kvuujv6dFBH3RcT2ag6IYaeOPn44Ih6q/u+tiIjXDkWd/VFHH/84In5U/R69ayTMgllESAOXAisycwawolruzl8B/zYoVTVOPf3bBPxuZs4C3gxcGhEHDWKN/VVPH58DzsvMI4DTgL+LiP0Gscb+qvff6RXAuYNWVT/VOcXvIuBXmfl64AvA5wa3yr6rs38/B84HvjW41TVGnX38IdCamUcDy4C/Gdwq+6fOPn4rM4+qfo/+DfD5QS6z4UoJ6QXAkur+EuDMrhpFxJuAycCtg1RXo/TYv8x8ITN/Uy3uQznvTb3q6eNPMnNtdf9xYAtQ16w7hajr32lmrgCeGayiGqCeKX47930ZMC8iuprQqEQ99i8zH83M+4GXhqLABqinj7dn5nPV4g+ozW0xnNTTx6c7Le5LFxNsDTelBMHkzNwEUN1O2rVBROwF/C3w0UGurRF67B9ARLw6Iu4HHgM+VwXZcFFXHztExHHA3sBPB6G2RulVH4eRg6n9m+uwoVrXZZvM3A48BRwwKNX1Xz39G+5628dFwHcHtKLGq6uPEfH+iPgptZH0fx2k2gZMf6YF7ZWI+BfgVV089Ik6N3EJ8H8y87ESP8A3oH9k5mPA0dVh7n+KiGWZublRNfZXI/pYbWcK8E1gYWYWNXJpVB+HmXqm+K1rGuBCDefa61V3HyPiHKAVOHlAK2q8eqeivhq4OiL+CPhzYOFAFzaQBi2kM/Ot3T0WEZsjYkpmbqp+gW/potkc4MSIuAQYC+wdEdsys4jvsW5A/zpv6/GIeBA4kdqhxSI0oo8RMQ74DvDnmfmDASq1zxr5Pg4j9Uzx29FmQ0SMBsYDvxyc8vqtGaYwrquPEfFWah84T+50em246O37eAPw5QGtaBCUcrj7Fn77aWchsHzXBpl5dma+JjOnAR8Bri0loOvQY/8iYmpEvKK6vz9wAvV9Y1gp6unj3sDN1N67mwaxtkbpsY/DVD1T/Hbu+zuAf83hMxNSM0xh3GMfI2I28BXgjMwcjh8w6+njjE6L84G1g1jfwMjMIf+hdm5rBbUXdAUwoVrfCny1i/bnA1cNdd2N7B9wCnA/8B/V7UVDXfcA9PEc4EVgdaefWUNdeyP7WC1/D2gHfk3t0/+pQ117HX37Q+An1K4R+ES17nJqv9ABxgA3AeuAe4FDhrrmBvfv2Oq9ehbYCjw41DUPQB//Bdjc6f/eLUNd8wD08YvAg1X/bgeOGOqa+/vjtKCSJBWqlMPdkiRpF4a0JEmFMqQlSSqUIS1JUqEMaUmSCmVIS5JUKENakqRC/X98dKaaTXuw5QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 504x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#在训练集上观察预测残差的分布，看是否符合模型假设：噪声为0均值的高斯噪声\n",
    "f, ax = plt.subplots(figsize=(7, 5)) \n",
    "f.tight_layout() \n",
    "ax.hist(y_train - y_train_pred_lr, bins=40, label='Residuals Linear', color='b', alpha=.5); \n",
    "ax.set_title(\"Histogram of Residuals\") \n",
    "ax.legend(loc='best');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "残差分布和高斯分布比较匹配，拟合较好。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.2 尝试训练正则化的线性回归（L2正则 --> 岭回归）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>coef</th>\n",
       "      <th>columns</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>0.204613</td>\n",
       "      <td>temp</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>0.177908</td>\n",
       "      <td>atemp</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>33</th>\n",
       "      <td>0.173104</td>\n",
       "      <td>yr</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>0.105251</td>\n",
       "      <td>weathersit_1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.088266</td>\n",
       "      <td>season_4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>0.078043</td>\n",
       "      <td>mnth_9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>0.044738</td>\n",
       "      <td>weathersit_2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>0.044606</td>\n",
       "      <td>mnth_5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>0.042529</td>\n",
       "      <td>mnth_6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>0.034348</td>\n",
       "      <td>mnth_3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>0.026175</td>\n",
       "      <td>weekday_6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>0.024455</td>\n",
       "      <td>workingday</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>0.023664</td>\n",
       "      <td>mnth_8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.012770</td>\n",
       "      <td>season_2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>0.012230</td>\n",
       "      <td>mnth_4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>0.009764</td>\n",
       "      <td>mnth_10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>0.009365</td>\n",
       "      <td>weekday_5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>0.007225</td>\n",
       "      <td>weekday_4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>0.006887</td>\n",
       "      <td>weekday_3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.000163</td>\n",
       "      <td>instant</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>-0.003928</td>\n",
       "      <td>weekday_2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-0.010514</td>\n",
       "      <td>season_3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>-0.016466</td>\n",
       "      <td>mnth_2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>-0.022072</td>\n",
       "      <td>weekday_0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>-0.022402</td>\n",
       "      <td>mnth_1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>-0.023651</td>\n",
       "      <td>weekday_1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>-0.027905</td>\n",
       "      <td>mnth_7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>-0.028558</td>\n",
       "      <td>holiday</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>-0.083505</td>\n",
       "      <td>mnth_11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-0.090522</td>\n",
       "      <td>season_1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>-0.094907</td>\n",
       "      <td>mnth_12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>-0.125146</td>\n",
       "      <td>windspeed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>-0.131431</td>\n",
       "      <td>hum</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>-0.149989</td>\n",
       "      <td>weathersit_3</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        coef       columns\n",
       "27  0.204613          temp\n",
       "28  0.177908         atemp\n",
       "33  0.173104            yr\n",
       "17  0.105251  weathersit_1\n",
       "4   0.088266      season_4\n",
       "13  0.078043        mnth_9\n",
       "18  0.044738  weathersit_2\n",
       "9   0.044606        mnth_5\n",
       "10  0.042529        mnth_6\n",
       "7   0.034348        mnth_3\n",
       "26  0.026175     weekday_6\n",
       "32  0.024455    workingday\n",
       "12  0.023664        mnth_8\n",
       "2   0.012770      season_2\n",
       "8   0.012230        mnth_4\n",
       "14  0.009764       mnth_10\n",
       "25  0.009365     weekday_5\n",
       "24  0.007225     weekday_4\n",
       "23  0.006887     weekday_3\n",
       "0   0.000163       instant\n",
       "22 -0.003928     weekday_2\n",
       "3  -0.010514      season_3\n",
       "6  -0.016466        mnth_2\n",
       "20 -0.022072     weekday_0\n",
       "5  -0.022402        mnth_1\n",
       "21 -0.023651     weekday_1\n",
       "11 -0.027905        mnth_7\n",
       "31 -0.028558       holiday\n",
       "15 -0.083505       mnth_11\n",
       "1  -0.090522      season_1\n",
       "16 -0.094907       mnth_12\n",
       "30 -0.125146     windspeed\n",
       "29 -0.131431           hum\n",
       "19 -0.149989  weathersit_3"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#岭回归／L2正则\n",
    "#class sklearn.linear_model.RidgeCV(alphas=(0.1, 1.0, 10.0), fit_intercept=True, \n",
    "#                                  normalize=False, scoring=None, cv=None, gcv_mode=None, \n",
    "#                                  store_cv_values=False)\n",
    "from sklearn.linear_model import  RidgeCV\n",
    "\n",
    "#1. 设置超参数（正则参数）范围\n",
    "alphas = [ 0.01, 0.1, 1, 10,100]\n",
    "#n_alphas = 20\n",
    "#alphas = np.logspace(-5,2,n_alphas)\n",
    "\n",
    "#2. 生成一个RidgeCV实例\n",
    "ridge = RidgeCV(alphas=alphas, store_cv_values=True)  \n",
    "\n",
    "#3. 模型训练\n",
    "ridge.fit(X_train, y_train)    \n",
    "\n",
    "#4. 预测\n",
    "y_test_pred_ridge = ridge.predict(X_test)\n",
    "y_train_pred_ridge = ridge.predict(X_train)\n",
    "\n",
    "# 看看各特征的权重系数，系数的绝对值大小可视为该特征的重要性\n",
    "fs = pd.DataFrame({ \"coef\":list((ridge.coef_.T)),\"columns\":list(feat_names)})\n",
    "fs.sort_values(by=['coef'],ascending=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The RMSE of LinearRegression on test is 0.09342398394396466\n",
      "The RMSE of LinearRegression on train is 0.08599183308329299\n"
     ]
    }
   ],
   "source": [
    "# 使用RMSE:mean_squared_error评价模型在测试集和训练集上的性能，并输出评估结果\n",
    "#测试集\n",
    "import math\n",
    "mset = ( mean_squared_error(y_test, y_test_pred_ridge))\n",
    "print ('The RMSE of LinearRegression on test is',math.sqrt(mset))\n",
    "#训练集\n",
    "msea= (mean_squared_error(y_train, y_train_pred_ridge))\n",
    "print ('The RMSE of LinearRegression on train is',math.sqrt(msea))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZIAAAEKCAYAAAA4t9PUAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xt4VfWd7/H3NwkJd1AI90tQ8BK0Ko1UBK2tFcHpFK9TtK0XmPH0nPGZaWc8Z5ynp/aMM/Oc41zqeTraznHGeGurtmo7tGUK9V6iXAKKEhCIJEC4hlsgQBKSfM8fa0U322Rnh2RlX/J5PU+erL3Wb6313YuQT9bea6+vuTsiIiJnKifVBYiISGZTkIiISLcoSEREpFsUJCIi0i0KEhER6RYFiYiIdEukQWJm88xss5lVmtkD7SwvMLMXwuWrzKwonD/CzF43s3oze7SDbS8xsw1R1i8iIp2LLEjMLBd4DJgPFAO3m1lx3LDFwGF3nwo8Ajwczm8Avgvc38G2bwbqo6hbRES6JsozkplApbtvc/cm4HlgQdyYBcDT4fSLwLVmZu5+3N1XEATKacxsMPAXwN9FV7qIiCQrL8Jtjwd2xjyuAT7X0Rh3bzazOmAEcCDBdv8W+GfgRLKFjBw50ouKipIdLiIiwNq1aw+4e2Fn46IMEmtnXvz9WJIZ88lgs0uBqe7+7bb3UxKMvRe4F2DSpEmUl5cnLFZERE5nZtuTGRflS1s1wMSYxxOA3R2NMbM8YBhwKME2ZwGfNbNqYAVwnpm90d5Ad3/c3UvcvaSwsNNAFRGRMxRlkKwBppnZFDPLBxYCS+LGLAHuCqdvBV7zBHeRdPcfufs4dy8C5gBb3P2aHq9cRESSFtlLW+F7HvcBy4BcoNTdK8zsIaDc3ZcATwDPmlklwZnIwrb1w7OOoUC+md0IzHX3jVHVKyIiZ8b6wm3kS0pKXO+RiIh0jZmtdfeSzsbpk+0iItItChIREekWBYmIiHSLgkREJAt9UFPHv7y6lfrG5sj3pSAREclCj/9+G4+/ta1X9qUgERHJMnvqTrL0gz189fKJDC6I8gYmAQWJiEiWefrt7bg7d11Z1Cv7U5CIiGSRE03NPLd6B9dPH8PEswf2yj4VJCIiWeTldbuoO3mKRXOm9No+FSQiIlmitdV5sqyKi8cPo2TyWb22XwWJiEiWeGtrLR/VHmfRnCLM2uvSEQ0FiYhIligtq6ZwSAF/cPG4Xt2vgkREJAts3XeMt7bUcucVk8nP691f7QoSEZEs8OTb1eTn5XDH5yb1+r4VJCIiGe7w8SZeXlfDzZeNZ8Tggl7fv4JERCTD/XT1DhpOtXLP7N675DeWgkREJIOdamnlmXeqmTN1JOePGZKSGhQkIiIZbOkHe9h3tJFFc4pSVoOCREQkQ7k7pSuqmDJyENecNypldShIREQy1LodR1hfU8c9s4vIyem9DyDGU5CIiGSo0rIqhvbP45YZE1Jah4JERCQD7Tpykt9u2MvtMycxqBd6jiSiIBERyUDPvF0NwJ291HMkEQWJiEiGOd4Y9ByZN30M44cPSHU5ChIRkUzz8roajjY0p/SS31gKEhGRDBL0HKnmkgnDmDGp93qOJKIgERHJIG9uqWXbgeMsmjOlV3uOJBJpkJjZPDPbbGaVZvZAO8sLzOyFcPkqMysK548ws9fNrN7MHo1b57dmtt7MKszsX80sN8rnICKSTkrLqhg9tID5F41NdSkfiyxIwl/wjwHzgWLgdjMrjhu2GDjs7lOBR4CHw/kNwHeB+9vZ9B+5+yXARUAhcFsE5YuIpJ0t+47x+60HuHNWUa/3HEkkykpmApXuvs3dm4DngQVxYxYAT4fTLwLXmpm5+3F3X0EQKKdx96PhZB6QD3gk1YuIpJnSFVUU5OVwx8ze7zmSSJRBMh7YGfO4JpzX7hh3bwbqgBGdbdjMlgH7gWMEASQiktUO1jfy8ru7uHnGBM4alJ/qck4TZZC09y5Q/NlDMmM+PcD9emAsUAB8sd2dm91rZuVmVl5bW9vZJkVE0tpzq3fQ1NzKotlFqS7lU6IMkhpgYszjCcDujsaYWR4wDDiUzMbdvQFYwqdfLmtb/ri7l7h7SWFhYRdLFxFJH03NrTzzznaumjaSaaNT03MkkSiDZA0wzcymmFk+sJDgF3+sJcBd4fStwGvu3uEZiZkNNrOx4XQecAPwYY9XLiKSRpZ+sIf9xxpZNCc1HRA7E9mdvty92czuA5YBuUCpu1eY2UNAubsvAZ4AnjWzSoIzkYVt65tZNTAUyDezG4G5wEFgiZkVhNt8DfjXqJ6DiEiquTulZVWcUziIz09Lz1dXIr1lpLsvBZbGzXswZrqBDi7fdfeiDjZ7eU/VJyKS7tZuP8z7NXX87Y0XpbTnSCLpcyGyiIh8yhMrqhg2oB+3zIi/6DV9KEhERNLUzkMnWFYR9BwZmJ/aniOJKEhERNLUM+9UY2bcOWtyqktJSEEiIpKG6hubeX7NTuZfNIZxadBzJBEFiYhIGnppbQ3HGprT9pLfWAoSEZE0E/QcqeLSicPTpudIIgoSEZE08/rm/VQfPJERZyOgIBERSTulZVWMHdaf+ReNSXUpSVGQiIikkU17jlJWeZA7ZxXRLzczfkVnRpUiIn3Ek2VV9O+Xw+0zJ3Y+OE0oSERE0sSB+kZ++d5ubpkxgeED06vnSCIKEhGRNPHTVUHPkXvSsOdIIgoSEZE00NjcwrMrt/P58wqZOir9eo4koiAREUkDv3l/D7Vp3HMkEQWJiEiKuTtPrKhi6qjBXD1tZKrL6TIFiYhIiq2pPkzF7qMsmj0Fs/TsOZKIgkREJMWeWLGN4QP7cdNl6dtzJBEFiYhICu04eILlG/dxx8xJDMjPTXU5Z0RBIiKSQk+/U02uGd9I854jiShIRERS5FjDKV5Ys5MbLh7L2GHp3XMkEQWJiEiKvLi2hvrGzOg5koiCREQkBVpanafermbGpOFcOnF4qsvpFgWJiEgKvPbhfrYfPMHiOeekupRuU5CIiKTAEyu2MW5Yf66fPjrVpXSbgkREpJdV7K5j5bZD3HVlEXkZ0nMkkcx/BiIiGebJsmoG9Mtl4eWTUl1Kj1CQiIj0otpjjSx5bze3fnYCwwb2S3U5PSLSIDGzeWa22cwqzeyBdpYXmNkL4fJVZlYUzh9hZq+bWb2ZPRozfqCZ/cbMPjSzCjP7P1HWLyLS036yajtNLa3cnWE9RxKJLEjMLBd4DJgPFAO3m1lx3LDFwGF3nwo8Ajwczm8Avgvc386m/8ndLwAuA2ab2fwo6hcR6WmNzS38eOV2vnB+IecWDk51OT0myjOSmUClu29z9ybgeWBB3JgFwNPh9IvAtWZm7n7c3VcQBMrH3P2Eu78eTjcB64AJET4HEZEe86v1ezhQ35TxH0CMF2WQjAd2xjyuCee1O8bdm4E6YEQyGzez4cAfAq92u1IRkYi5O6Urqjhv9GDmTM28niOJRBkk7d1U389gzKc3bJYHPAf8wN23dTDmXjMrN7Py2traTosVEYnSym2H2Lgnc3uOJBJlkNQAE2MeTwB2dzQmDIdhwKEktv04sNXd/29HA9z9cXcvcfeSwsLCLhUuItLTSsuqOGtgP27M0J4jiUQZJGuAaWY2xczygYXAkrgxS4C7wulbgdfcPeEZiZn9HUHgfKuH6xURicT2g8d5ZdM+vva5yfTvl5k9RxLJi2rD7t5sZvcBy4BcoNTdK8zsIaDc3ZcATwDPmlklwZnIwrb1zawaGArkm9mNwFzgKPAd4ENgXXh6+Ki7/3tUz0NEpLueejvze44kElmQALj7UmBp3LwHY6YbgNs6WLeog81m14uLIpLVjjWc4uflNXz5M2MZPbR/qsuJhD7ZLiISoZ+VZ0fPkUQUJCIiEQl6jlRxedFZfGZCZvccSURBIiISkd9t3MfOQydZNDt7z0ZAQSIiEpnSsirGDx/AdcWZ33MkEQWJiEgENuyqY3XVIe7Okp4jiWT3sxMRSZHSsioG5ufyR5dP7HxwhlOQiIj0sP3HGvjV+t3c9tkJDBuQHT1HElGQiIj0sB+v3MGpFufuLH+TvY2CRESkBzWcauEnK7dz7QWjmDJyUKrL6RUKEhGRHrRk/W4OHm9icRZ/ADGegkREpIe09Ry5YMwQZp2bVGulrKAgERHpIe98dJAP9x7Lyp4jiShIRER6SGlZFWcPyucrl45LdSm9SkEiItIDqg4c59UP9/P1z03Kyp4jiShIRER6wNNvV5OXY3z9iuzsOZKIgkREpJvqTp7iZ+U7+cPPjGNUlvYcSURBIiLSTT8v38mJppas7jmSiIJERKQbmltaebKsmplTzuai8cNSXU5KKEhERLrhdxv3setI9vccSURBIiLSDaVlVUw4K/t7jiSiIBEROUPv1xxhTfVh7r6yiNycvvMBxHgKEhGRM/RkWTWD+kjPkUQUJCIiZ2Df0QZ+/f5ubiuZyND+2d9zJJGkg8TM5pjZPeF0oZn13XeWRKTP+/HK7TS3OvfMLkp1KSmXVJCY2feAvwL+OpzVD/hxVEWJiKSzhlMt/GTVDr504Wgmj+gbPUcSSfaM5CbgK8BxAHffDQyJqigRkXT2y3d3ceh4U5++5DdWskHS5O4OOICZKYJFpE9yd0rLqrhw7FCuOOfsVJeTFpINkp+Z2f8DhpvZnwCvAP/W2UpmNs/MNptZpZk90M7yAjN7IVy+ysyKwvkjzOx1M6s3s0fj1vl7M9tpZvVJ1i4i0mPKKg+yZV89i2YX9ameI4kkFSTu/k/Ai8BLwPnAg+7+L4nWMbNc4DFgPlAM3G5mxXHDFgOH3X0q8AjwcDi/AfgucH87m/4VMDOZukVEelppWRUjB+fzh5f0rZ4jiST7Zvsg4DV3/+8EZyIDzKyz691mApXuvs3dm4DngQVxYxYAT4fTLwLXmpm5+3F3X0EQKKdx95XuvieZukVEetK22npe+3A/X/vc5D7XcySRZF/aegsoMLPxBC9r3QM81ck644GdMY9rwnntjnH3ZqAO6DuNjkUkozz1djX5uTl9sudIIskGibn7CeBm4F/c/SaCl6sSrtPOPD+DMWfEzO41s3IzK6+tre2JTYpIH1Z34hQ/L6/hK5eOo3BIQarLSStJB4mZzQK+BvwmnJfXyTo1QOx9AyYAuzsaY2Z5wDDgUJI1JeTuj7t7ibuXFBYW9sQmRaQPe6F8BydPtegDiO1INkj+HHgAeNndK8JPtb/WyTprgGlmNsXM8oGFwJK4MUuAu8LpWwneh+mRMxIRkZ7S3NLK029v54pzzmb6uL7ZcySRZIPkBNBKcOXV+wQB8IVEK4TvedwHLAM2AT8LQ+ghM/tKOOwJYISZVQJ/QRBWAJhZNfB94G4zq2m74svM/sHMaoCB4fz/leRzEBE5I8sq1HMkEUvmBMDMNhNciruBIFAAcPft0ZXWc0pKSry8vDzVZYhIhrrlR29Te6yR1++/pk/dLt7M1rp7SWfjOnufo02tu/+qmzWJiGSc93YeYe32wzz45eI+FSJdkWyQfM/M/h14FWhsm+nuL0dSlYhImniyrIrBBXncVjIh1aWkrWSD5B7gAoK7/ra9tOWAgkREstbeugZ+8/4e7pxVxJA+3nMkkWSD5BJ3vzjSSkRE0syzK6tpdfUc6UyyV22tbOc+WSIiWetkU9Bz5Lri0Uw8e2Cqy0lryZ6RzAHuMrMqgvdIDHB3/0xklYmIpNAv3t3FkROndMlvEpINknmRViEikkbaeo5MHzeUmVPUc6QzSQVJpnxeRESkJ/x+6wEq99fzz7ddop4jSUj2PRIRkT4j6DlSwJcvGZvqUjKCgkREJEbl/nre2FzLN66YTEGeeo4kQ0EiIhLjqberyM/L4WtXTEp1KRlDQSIiEjpyoomX1u7ixkvHMXKweo4kS0EiIhJ6bvXOsOeILvntCgWJiAhwqqWVZ96p5spzR3Dh2KGpLiejKEhERIDfbtjLnroGfQDxDChIREQILvmdPGIgX7xgVKpLyTgKEhHp89btOMy7O45wz5VF5KjnSJcpSESkz3uyrJohBXncWjIx1aVkJAWJiPRpe+pOsvSDPSycOZHBBcneflBiKUhEpE975p3tuDt3zipKdSkZS0EiIn3WiaZmfrpqB9dPH6OeI92gIBGRPuvldbuoO3mKRXN0yW93KEhEpE9qbXWeLKvi4vHDKJl8VqrLyWgKEhHpk97aWstHtcdZNKdIPUe6SUEiIn1SaVk1o4YU8AcXj0t1KRlPQSIifc7Wfcd4a0std86aTH6efg12l46giPQ5T75dTUFeDrfPVM+RnhBpkJjZPDPbbGaVZvZAO8sLzOyFcPkqMysK548ws9fNrN7MHo1b57Nm9kG4zg9ML26KSBccPt7Ey+tquOmy8YxQz5EeEVmQmFku8BgwHygGbjez4rhhi4HD7j4VeAR4OJzfAHwXuL+dTf8IuBeYFn7N6/nqRSRb/XT1DhpOtarnSA+K8oxkJlDp7tvcvQl4HlgQN2YB8HQ4/SJwrZmZux939xUEgfIxMxsLDHX3d9zdgWeAGyN8DiKSRdp6jsyZOpLzxwxJdTlZI8ogGQ/sjHlcE85rd4y7NwN1wIhOtlnTyTZFRNq19IM97DvayKI5RakuJatEGSTtvXfhZzDmjMab2b1mVm5m5bW1tQk2KSJ9gbtTuqKKKSMHcc156jnSk6IMkhog9p7ME4DdHY0xszxgGHCok21O6GSbALj74+5e4u4lhYWFXSxdRLLNuh1HWF9Txz2z1XOkp0UZJGuAaWY2xczygYXAkrgxS4C7wulbgdfC9z7a5e57gGNmdkV4tdadwH/0fOkikm1Ky6oY2j+PW2ZM6HywdElkN99392Yzuw9YBuQCpe5eYWYPAeXuvgR4AnjWzCoJzkQWtq1vZtXAUCDfzG4E5rr7RuC/Ak8BA4D/DL9ERDq068hJfrthL388ZwqD1HOkx0V6RN19KbA0bt6DMdMNwG0drFvUwfxy4KKeq1JEst0zb1cDcOeVRSmtI1vpk+0iktWONzbz3OodzJs+hvHDB6S6nKykIBGRrPbyuhqONjTrkt8IKUhEJGsFPUequWTCMGZMUs+RqChIRCRrvbmllm0HjrNozhT1HImQgkREslZpWRWjhxZww8VjU11KVlOQiEhW2rLvGL/feoA7ZxXRL1e/6qKkoysiWal0RRUFeTncoZ4jkVOQiEjWOVjfyMvv7uLmGRM4a1B+qsvJegoSEck6z63eQVNzK4tmF6W6lD5BQSIiWaWpuZVn3tnOVdNGMm20eo70BgWJiGSVpR/sYf+xRhbNUQfE3qIgEZGs4e6UllVxbuEgPj9N7SN6i4JERLLG2u2Heb+mjntmT1HPkV6kIBGRrFFaVsWwAf24eYY6cPcmBYmIZIWdh07w2w17uX3mJAbmq+dIb1KQiEhWeOadasyMO2dNTnUpfY6CREQyXn1jM8+v2cn8i8YwTj1Hep2CREQy3ktrazjW0KxLflNEQSIiGS3oOVLFpROHq+dIiihIRCSjvb55P9UHT7BYZyMpoyARkYxWWlbF2GH9mXfRmFSX0mcpSEQkY23ac5SyyoPqOZJiOvIikrGeLKuif78cbp85MdWl9GkKEhHJSAfqG/nle7u5ZcYEhg9Uz5FUUpCISEb66aqg58g96jmScgoSEck4jc0tPLtyO58/r5Cpo9RzJNUUJCKScX7z/h5qjzXqkt80EWmQmNk8M9tsZpVm9kA7ywvM7IVw+SozK4pZ9tfh/M1mdn3M/D83sw1mVmFm34qyfhFJP+7OEyuqmDpqMFdNG5nqcoQIg8TMcoHHgPlAMXC7mRXHDVsMHHb3qcAjwMPhusXAQmA6MA/4oZnlmtlFwJ8AM4FLgC+b2bSonoOIpJ811Yep2H2URbOnYKaeI+kgyjOSmUClu29z9ybgeWBB3JgFwNPh9IvAtRb8ZCwAnnf3RnevAirD7V0IrHT3E+7eDLwJ3BThcxCRNPPEim0MH9iPmy5Tz5F0EWWQjAd2xjyuCee1OyYMhjpgRIJ1NwBXm9kIMxsI3ADoAnKRPmLHwRMs37iPO2ZOYkB+bqrLkVCU3V/aO+f0JMe0O9/dN5nZw8DvgHpgPdDc7s7N7gXuBZg0aVKyNYtIGnv6nWpyzfiGeo6klSjPSGo4/WxhArC7ozFmlgcMAw4lWtfdn3D3Ge5+dTh2a3s7d/fH3b3E3UsKCwt74OmISCodazjFC2t2csPFYxk7TD1H0kmUQbIGmGZmU8wsn+DN8yVxY5YAd4XTtwKvubuH8xeGV3VNAaYBqwHMbFT4fRJwM/BchM9BRFKs4VQLr2zcx7dfWE99o3qOpKPIXtpy92Yzuw9YBuQCpe5eYWYPAeXuvgR4AnjWzCoJzi4WhutWmNnPgI0EL139qbu3hJt+ycxGAKfC+Yejeg4ikhp1J0/x+of7WVaxlze31HKiqYUhBXn8l6vP4dKJw1NdnsSx4AQgu5WUlHh5eXmqyxCRBPYdbWD5xn0sr9jLOx8dpLnVKRxSwHXFo7l++hhmnTOC/Dx9hro3mdlady/pbFyUb7aLiCT0UW09yyr2srxiH+/tPAJA0YiBLJ4zhbnTx3DZxOHk5OizIulOQSIivcbdeb+mLgiPjfuo3F8PwMXjh/GX153H9ReNYdqowfqgYYZRkIhIpE61tLK66tDHZx57jzaQm2PMLDqbr39uEtdNH8P44boKK5MpSESkx51oauatLQdYXrGXVz/cT93JUxTk5XD1eYXcP/18rr1gFGcNUg+RbKEgEZEecfh4E69s2sfyjfv4/dZaGk61MmxAP669cBRzi8dw9XkjGZivXznZSP+qInLGdh05yfLwJavV1YdoaXXGDuvPV0smMnf6GGZOOVu91PsABYmIJM3d2bq/nmUb9rJs41427DoKwNRRg/nm58/h+uljuHj8ML1Z3scoSEQkodZW592dh1lesY9lFXupPngCgMsmDeev5l3A3OmjObdwcIqrlFRSkIjIpzQ1t/L2RwdYVrGP323cx4H6RvJyjFnnjmDxVecwt3g0o4f2T3WZkiYUJCICQH1jM29s3s+yin288eF+jjU2MzA/l2vOL+T66WO45vxRDBvQL9VlShpSkIj0YbXHGnl1U/CSVVnlQZpaWjl7UD7zLx7D9dPHMHvqSPr3U98PSUxBItLHBM2h9rKsYi/l2w/jDhPOGsA3Zk1mbvFoSorOJle3JZEuUJCIZDl3Z+OeoyyrCG6I+OHeYwBcMGYIf/bFacydPprisUN1pZWcMQWJSBZqaXXKqw8F4bFxLzWHT2IGJZPP4n/+wYXMLR7DpBEDU12mZAkFiUiWaDjVQlnlAZZV7OWVTfs5dLyJ/NwcZk8dwX1fmMqXikczcnBBqsuULKQgEclgbQ2glm/cyxubP2kA9YULRjF3+miuOX8Ugwv031yipZ8wkQzTUQOoGy8bz9zi0cw6dwQFebrSSnqPgkQkA2yrrf/4/Y53d6gBlKQXBUkCr2/eT0NTC/l5OcFXbs7H0wV5OeTn5tIvz06bn5+bo6tfpNvcnQ92BQ2gllWoAZSkNwVJAn/36418VHu8y+vFB0vbdL9wuqCD5W2PC2LGth9gpy/rF7Neu9tVuGWEtgZQy8PugXvq1ABKMoOCJIEn757J8aZmmppbaWppDb43t9IY9/hU23RLuKztq6UlZvqTdU+1tHKiqZkjJ2PGxq/f0op7zz2X0wPNYsImN2G49YsNqE7CrV9uDnm5hpmRY2AE3zHIMcOAnJzgu5lhsfPDx9a2Xs4n6wfzTx+XEwZj2/Y+tX64TlsdlpNgX7FjezlwTza18OaWWpZv3Murm05vAPWXc9UASjKDgiSBVF5n7+40t/onIdVOgMWHW2ygtRtOMeF2qsXjttnSYbjFfm/twXBLV0HAxAbOp0PM+CTgYoOImPD7ZFzMenHr7zpy8pMGUBeMYu50NYCSzKOf1jRlZvTLNfrl5jAwjf4gbW5JfHbW3OqA0+rgDq3uuAfB6HzyuDV87B8/DqaDoIpb/7RxCdYP18P9tO19an1OryvhvsKxnLbvYF+nrR9Tc8L1+eS5usNV0wq5rni0GkBJRlOQSJfk5eaQl2bhJiKppT+BRESkWxQkIiLSLQoSERHplkiDxMzmmdlmM6s0swfaWV5gZi+Ey1eZWVHMsr8O5282s+tj5n/bzCrMbIOZPWdm6vcpIpJCkQWJmeUCjwHzgWLgdjMrjhu2GDjs7lOBR4CHw3WLgYXAdGAe8EMzyzWz8cCfASXufhGQG44TEZEUifKMZCZQ6e7b3L0JeB5YEDdmAfB0OP0icK0FnwhbADzv7o3uXgVUhtuD4EqzAWaWBwwEdkf4HEREpBNRBsl4YGfM45pwXrtj3L0ZqANGdLSuu+8C/gnYAewB6tx9eSTVi4hIUqIMkvbuNRH/ueiOxrQ738zOIjhbmQKMAwaZ2dfb3bnZvWZWbmbltbW1XShbRES6IsoPJNYAE2MeT+DTL0O1jakJX6oaBhxKsO6XgCp3rwUws5eBK4Efx+/c3R8HHg/H1ZrZ9jN8HiOBA2e4bpRUV9eorq5RXV2TrXVNTmZQlEGyBphmZlOAXQRvit8RN2YJcBfwDnAr8Jq7u5ktAX5qZt8nOPOYBqwGWoErzGwgcBK4FijvrBB3LzzTJ2Fm5e5ecqbrR0V1dY3q6hrV1TV9va7IgsTdm83sPmAZwdVVpe5eYWYPAeXuvgR4AnjWzCoJzkQWhutWmNnPgI1AM/Cn7t4CrDKzF4F14fx3Cc86REQkNSK915a7LwWWxs17MGa6Abitg3X/Hvj7duZ/D/hez1YqIiJnSp9s71y6nvGorq5RXV2jurqmT9dl3pPdk0REpM/RGYmIiHSLgiSOmf2jmX1oZu+b2S/MbHgH4xLeRyyCum4L7zHWamYdXoVhZtVm9oGZvWdmnV7R1ot19fbxOtvMfmdmW8PvZ3UwriU8Vu+FVwtGVc8Z33cuSknUdXd4+XzbMfrjXqip1Mz2m9mGDpabmf0grPl9M5sRdU1J1nWNmdXFHKsH2xsXQV0Tzex1M9sU/l/883bGRHvMgk5t+mr7AuYCeeH0w8DD7YwoQnjrAAAGP0lEQVTJBT4CzgHygfVAccR1XQicD7xBcK+xjsZVAyN78Xh1WleKjtc/AA+E0w+09+8YLqvvhWPU6fMH/hvwr+H0QuCFNKnrbuDR3vp5Cvd5NTAD2NDB8huA/yT44PIVwKo0qesa4Ne9eazC/Y4FZoTTQ4At7fw7RnrMdEYSx92Xe3C7FoCVBB+GjJfMfcR6uq5N7r45yn2ciSTr6vXjxen3cXsauDHi/SXSnfvOpbquXufubxF8HKAjC4BnPLASGG5mY9OgrpRw9z3uvi6cPgZs4tO3o4r0mClIEltEkOLxkrmPWKo4sNzM1prZvakuJpSK4zXa3fdA8B8NGNXBuP7hrXRWmllUYdOd+85FKdl/l1vCl0NeNLOJ7Szvben8/2+Wma03s/80s+m9vfPwJdHLgFVxiyI9Zn2yZ7uZvQKMaWfRd9z9P8Ix3yH40ONP2ttEO/O6fflbMnUlYba77zazUcDvzOzD8C+pVNbV68erC5uZFB6vc4DXzOwDd/+ou7XF6c5956KUzD5/BTzn7o1m9k2Cs6YvRlxXZ1JxrJKxDpjs7vVmdgPwS4K7cvQKMxsMvAR8y92Pxi9uZ5UeO2Z9Mkjc/UuJlpvZXcCXgWs9fIExTjL3EevxupLcxu7w+34z+wXByxfdCpIeqKvXj5eZ7TOzse6+JzyF39/BNtqO1zYze4Pgr7meDpLu3HcuSp3W5e4HYx7+G2HPoBSL5Oepu2J/ebv7UjP7oZmNdPfI78FlZv0IQuQn7v5yO0MiPWZ6aSuOmc0D/gr4iruf6GDYx/cRM7N8gjdHI7viJ1lmNsjMhrRNE1w40O4VJr0sFcer7T5uhN8/deZkZmeZWUE4PRKYTXBbnp6WzPOPrffj+85FUEuX6op7Hf0rBK+/p9oS4M7wSqQrCNpJ7El1UWY2pu19LTObSfD79WDitXpkv0Zwu6lN7v79DoZFe8x6+wqDdP8iaKK1E3gv/Gq7kmYcsDRm3A0EV0d8RPAST9R13UTwV0UjsA9YFl8XwdU368OvinSpK0XHawTwKrA1/H52OL8E+Pdw+krgg/B4fQAsjrCeTz1/4CGCP1gA+gM/D3/+VgPnRH2Mkqzrf4c/S+uB14ELeqGm5wj6DZ0Kf7YWA98EvhkuN4Luqx+F/24dXsXYy3XdF3OsVgJX9lJdcwhepno/5vfWDb15zPTJdhER6Ra9tCUiIt2iIBERkW5RkIiISLcoSEREpFsUJCIi0i0KEpEEzKy+m+u/GH5qPtGYNyzBnZOTHRM3vtDMfpvseJHuUJCIRCS811Kuu2/r7X27ey2wx8xm9/a+pe9RkIgkIfxE8D+a2QYL+r18NZyfE94Ko8LMfm1mS83s1nC1rxHziXoz+1F4g8gKM/ubDvZTb2b/bGbrzOxVMyuMWXybma02sy1mdlU4vsjMfh+OX2dmV8aM/2VYg0ikFCQiybkZuBS4BPgS8I/h7UNuBoqAi4E/BmbFrDMbWBvz+DvuXgJ8Bvi8mX2mnf0MAta5+wzgTeB7Mcvy3H0m8K2Y+fuB68LxXwV+EDO+HLiq609VpGv65E0bRc7AHIK74LYA+8zsTeDycP7P3b0V2Gtmr8esMxaojXn8R+Gt/fPCZcUEt7WI1Qq8EE7/GIi9AV/b9FqC8ALoBzxqZpcCLcB5MeP3E9yqRiRSChKR5HTUZCpR86mTBPfQwsymAPcDl7v7YTN7qm1ZJ2LvYdQYfm/hk/+73ya4x9klBK8wNMSM7x/WIBIpvbQlkpy3gK+aWW74vsXVBDdXXEHQ+CnHzEYTtFttswmYGk4PBY4DdeG4+R3sJ4fg7r8Ad4TbT2QYsCc8I/oGQfvcNueRHnd/liynMxKR5PyC4P2P9QRnCf/D3fea2UvAtQS/sLcQdKarC9f5DUGwvOLu683sXYK7w24DyjrYz3FgupmtDbfz1U7q+iHwkpndRnB33uMxy74Q1iASKd39V6SbzGywB13xRhCcpcwOQ2YAwS/32eF7K8lsq97dB/dQXW8BC9z9cE9sT6QjOiMR6b5fm9lwIB/4W3ffC+DuJ83sewS9sXf0ZkHhy2/fV4hIb9AZiYiIdIvebBcRkW5RkIiISLcoSEREpFsUJCIi0i0KEhER6RYFiYiIdMv/B2nfDsAgVaE1AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "alpha is: 1.0\n"
     ]
    }
   ],
   "source": [
    "mse_mean = np.mean(ridge.cv_values_, axis = 0)\n",
    "plt.plot(np.log10(alphas), mse_mean.reshape(len(alphas),1)) \n",
    "\n",
    "plt.xlabel('log(alpha)')\n",
    "plt.ylabel('mse')\n",
    "plt.show()\n",
    "#打印叉验证最佳alpha值：\n",
    "print ('alpha is:', ridge.alpha_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>coef_lr</th>\n",
       "      <th>coef_ridge</th>\n",
       "      <th>columns</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>33</th>\n",
       "      <td>0.523551</td>\n",
       "      <td>0.173104</td>\n",
       "      <td>yr</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>0.305429</td>\n",
       "      <td>0.204613</td>\n",
       "      <td>temp</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>0.148181</td>\n",
       "      <td>0.078043</td>\n",
       "      <td>mnth_9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>0.114507</td>\n",
       "      <td>0.177908</td>\n",
       "      <td>atemp</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>0.106982</td>\n",
       "      <td>0.009764</td>\n",
       "      <td>mnth_10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>0.105201</td>\n",
       "      <td>0.105251</td>\n",
       "      <td>weathersit_1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.095557</td>\n",
       "      <td>0.088266</td>\n",
       "      <td>season_4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>0.067452</td>\n",
       "      <td>-0.094907</td>\n",
       "      <td>mnth_12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>0.059572</td>\n",
       "      <td>0.023664</td>\n",
       "      <td>mnth_8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>0.047123</td>\n",
       "      <td>0.044738</td>\n",
       "      <td>weathersit_2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>0.046841</td>\n",
       "      <td>-0.083505</td>\n",
       "      <td>mnth_11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>0.027430</td>\n",
       "      <td>0.026175</td>\n",
       "      <td>weekday_6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>0.024960</td>\n",
       "      <td>0.024455</td>\n",
       "      <td>workingday</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>0.021836</td>\n",
       "      <td>0.042529</td>\n",
       "      <td>mnth_6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.009795</td>\n",
       "      <td>0.012770</td>\n",
       "      <td>season_2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>0.008918</td>\n",
       "      <td>0.009365</td>\n",
       "      <td>weekday_5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>0.007647</td>\n",
       "      <td>0.006887</td>\n",
       "      <td>weekday_3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>0.005022</td>\n",
       "      <td>0.007225</td>\n",
       "      <td>weekday_4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>0.000576</td>\n",
       "      <td>0.044606</td>\n",
       "      <td>mnth_5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>-0.000796</td>\n",
       "      <td>0.000163</td>\n",
       "      <td>instant</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>-0.003675</td>\n",
       "      <td>-0.003928</td>\n",
       "      <td>weekday_2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-0.015049</td>\n",
       "      <td>-0.010514</td>\n",
       "      <td>season_3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>-0.022045</td>\n",
       "      <td>-0.022072</td>\n",
       "      <td>weekday_0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>-0.023297</td>\n",
       "      <td>-0.023651</td>\n",
       "      <td>weekday_1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>-0.023371</td>\n",
       "      <td>-0.027905</td>\n",
       "      <td>mnth_7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>-0.030345</td>\n",
       "      <td>-0.028558</td>\n",
       "      <td>holiday</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>-0.055881</td>\n",
       "      <td>0.012230</td>\n",
       "      <td>mnth_4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>-0.062292</td>\n",
       "      <td>0.034348</td>\n",
       "      <td>mnth_3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-0.090303</td>\n",
       "      <td>-0.090522</td>\n",
       "      <td>season_1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>-0.135164</td>\n",
       "      <td>-0.125146</td>\n",
       "      <td>windspeed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>-0.138876</td>\n",
       "      <td>-0.016466</td>\n",
       "      <td>mnth_2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>-0.149401</td>\n",
       "      <td>-0.131431</td>\n",
       "      <td>hum</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>-0.152324</td>\n",
       "      <td>-0.149989</td>\n",
       "      <td>weathersit_3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>-0.171022</td>\n",
       "      <td>-0.022402</td>\n",
       "      <td>mnth_1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     coef_lr  coef_ridge       columns\n",
       "33  0.523551    0.173104            yr\n",
       "27  0.305429    0.204613          temp\n",
       "13  0.148181    0.078043        mnth_9\n",
       "28  0.114507    0.177908         atemp\n",
       "14  0.106982    0.009764       mnth_10\n",
       "17  0.105201    0.105251  weathersit_1\n",
       "4   0.095557    0.088266      season_4\n",
       "16  0.067452   -0.094907       mnth_12\n",
       "12  0.059572    0.023664        mnth_8\n",
       "18  0.047123    0.044738  weathersit_2\n",
       "15  0.046841   -0.083505       mnth_11\n",
       "26  0.027430    0.026175     weekday_6\n",
       "32  0.024960    0.024455    workingday\n",
       "10  0.021836    0.042529        mnth_6\n",
       "2   0.009795    0.012770      season_2\n",
       "25  0.008918    0.009365     weekday_5\n",
       "23  0.007647    0.006887     weekday_3\n",
       "24  0.005022    0.007225     weekday_4\n",
       "9   0.000576    0.044606        mnth_5\n",
       "0  -0.000796    0.000163       instant\n",
       "22 -0.003675   -0.003928     weekday_2\n",
       "3  -0.015049   -0.010514      season_3\n",
       "20 -0.022045   -0.022072     weekday_0\n",
       "21 -0.023297   -0.023651     weekday_1\n",
       "11 -0.023371   -0.027905        mnth_7\n",
       "31 -0.030345   -0.028558       holiday\n",
       "8  -0.055881    0.012230        mnth_4\n",
       "7  -0.062292    0.034348        mnth_3\n",
       "1  -0.090303   -0.090522      season_1\n",
       "30 -0.135164   -0.125146     windspeed\n",
       "6  -0.138876   -0.016466        mnth_2\n",
       "29 -0.149401   -0.131431           hum\n",
       "19 -0.152324   -0.149989  weathersit_3\n",
       "5  -0.171022   -0.022402        mnth_1"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 看看各特征的权重系数，系数的绝对值大小可视为该特征的重要性\n",
    "fs = pd.DataFrame({\"coef_lr\":list((lr.coef_.T)), \"coef_ridge\":list((ridge.coef_.T)),\"columns\":list(feat_names)})\n",
    "fs.sort_values(by=['coef_lr'],ascending=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.3 尝试训练正则化的线性回归（L1正则 --> Lasso）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>coef</th>\n",
       "      <th>columns</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.000628</td>\n",
       "      <td>instant</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>-0.000000</td>\n",
       "      <td>weathersit_2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>workingday</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>-0.000000</td>\n",
       "      <td>holiday</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>-0.000000</td>\n",
       "      <td>windspeed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>-0.000000</td>\n",
       "      <td>hum</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>atemp</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>temp</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>-0.000000</td>\n",
       "      <td>weekday_6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>weekday_5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>weekday_4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>weekday_3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>weekday_2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>-0.000000</td>\n",
       "      <td>weekday_1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>-0.000000</td>\n",
       "      <td>weekday_0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>-0.000000</td>\n",
       "      <td>weathersit_3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>weathersit_1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>mnth_5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>mnth_4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>season_2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>season_3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>-0.000000</td>\n",
       "      <td>season_4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>-0.000000</td>\n",
       "      <td>mnth_1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>-0.000000</td>\n",
       "      <td>mnth_2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>mnth_3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>33</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>yr</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>-0.000000</td>\n",
       "      <td>mnth_12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>mnth_6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>mnth_7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>mnth_8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>mnth_9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>mnth_10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>-0.000000</td>\n",
       "      <td>mnth_11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-0.039253</td>\n",
       "      <td>season_1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        coef       columns\n",
       "0   0.000628       instant\n",
       "18 -0.000000  weathersit_2\n",
       "32  0.000000    workingday\n",
       "31 -0.000000       holiday\n",
       "30 -0.000000     windspeed\n",
       "29 -0.000000           hum\n",
       "28  0.000000         atemp\n",
       "27  0.000000          temp\n",
       "26 -0.000000     weekday_6\n",
       "25  0.000000     weekday_5\n",
       "24  0.000000     weekday_4\n",
       "23  0.000000     weekday_3\n",
       "22  0.000000     weekday_2\n",
       "21 -0.000000     weekday_1\n",
       "20 -0.000000     weekday_0\n",
       "19 -0.000000  weathersit_3\n",
       "17  0.000000  weathersit_1\n",
       "9   0.000000        mnth_5\n",
       "8   0.000000        mnth_4\n",
       "2   0.000000      season_2\n",
       "3   0.000000      season_3\n",
       "4  -0.000000      season_4\n",
       "5  -0.000000        mnth_1\n",
       "6  -0.000000        mnth_2\n",
       "7   0.000000        mnth_3\n",
       "33  0.000000            yr\n",
       "16 -0.000000       mnth_12\n",
       "10  0.000000        mnth_6\n",
       "11  0.000000        mnth_7\n",
       "12  0.000000        mnth_8\n",
       "13  0.000000        mnth_9\n",
       "14  0.000000       mnth_10\n",
       "15 -0.000000       mnth_11\n",
       "1  -0.039253      season_1"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#### Lasso／L1正则\n",
    "# class sklearn.linear_model.LassoCV(eps=0.001, n_alphas=100, alphas=None, fit_intercept=True, \n",
    "#                                    normalize=False, precompute=’auto’, max_iter=1000, \n",
    "#                                    tol=0.0001, copy_X=True, cv=None, verbose=False, n_jobs=1,\n",
    "#                                    positive=False, random_state=None, selection=’cyclic’)\n",
    "from sklearn.linear_model import LassoCV\n",
    "\n",
    "#1. 设置超参数搜索范围\n",
    "alphas = [ 0.01, 0.1, 1, 10,100]\n",
    "\n",
    "#2. 生成学习器实例\n",
    "lasso = LassoCV(alphas=alphas)\n",
    "\n",
    "#1. 设置超参数搜索范围\n",
    "#Lasso可以自动确定最大的alpha，所以另一种设置alpha的方式是设置最小的alpha值（eps） 和 超参数的数目（n_alphas），\n",
    "#然后LassoCV对最小值和最大值之间在log域上均匀取值n_alphas个\n",
    "# np.logspace(np.log10(alpha_max * eps), np.log10(alpha_max),num=n_alphas)[::-1]\n",
    "\n",
    "#2 生成LassoCV实例（默认超参数搜索范围）\n",
    "lasso = LassoCV()  \n",
    "\n",
    "#3. 训练（内含CV）\n",
    "lasso.fit(X_train, y_train)  \n",
    "\n",
    "#4.  预测模型\n",
    "y_test_pred_lasso = lasso.predict(X_test)\n",
    "y_train_pred_lasso = lasso.predict(X_train)\n",
    "\n",
    "# 看看各特征的权重系数，系数的绝对值大小可视为该特征的重要性\n",
    "fs = pd.DataFrame({ \"coef\":list((lasso.coef_.T)),\"columns\":list(feat_names)})\n",
    "fs.sort_values(by=['coef'],ascending=False)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The RMSE of LinearRegression on test is 0.1643648008616637\n",
      "The RMSE of LinearRegression on train is 0.16561621048101285\n"
     ]
    }
   ],
   "source": [
    "# 使用RMSE:mean_squared_error评价模型在测试集和训练集上的性能，并输出评估结果\n",
    "#测试集\n",
    "import math\n",
    "mset = ( mean_squared_error(y_test, y_test_pred_lasso))\n",
    "print ('The RMSE of LinearRegression on test is',math.sqrt(mset))\n",
    "#训练集\n",
    "msea= (mean_squared_error(y_train, y_train_pred_lasso))\n",
    "print ('The RMSE of LinearRegression on train is',math.sqrt(msea))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZgAAAEKCAYAAAAvlUMdAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xt4XdV55/HvT5IlGV+xLRtjG9tgc7EpGCJMCJCUcCnQDMYNTEymQDM0hEmZJtNkJiSZSYFpZwbahM4USgqBCUMbICHQOA2UECANIWAsE18xxsLYWL7KyBfJN93e+eNswUFI1rGlrXN09Ps8z3nO3muvvfa7fNGrtS9rKyIwMzPrayX5DsDMzIqTE4yZmaXCCcbMzFLhBGNmZqlwgjEzs1Q4wZiZWSqcYMzMLBVOMGZmlgonGDMzS0VZvgPIp3HjxsW0adPyHYaZ2YCyZMmSHRFR1VO9QZ1gpk2bRk1NTb7DMDMbUCRtyKWeT5GZmVkqnGDMzCwVTjBmZpYKJxgzM0uFE4yZmaXCCcbMzFLhBGNmZqlwgjEzG0Ta24O//NnrLK/blfqxnGDMzAaRN7c3cv+Lb7N2W1Pqx3KCMTMbRGrW7wSgetrRqR/LCcbMbBBZsmEn44ZXcNyYo1I/lhOMmdkgUrOhgeqpRyMp9WOlmmAkXSppjaRaSbd0sb1C0mPJ9kWSpnXafpykJklfTdZPkrQ067NH0peTbbdK2pS17fI0+2ZmNtBs33OAjQ37++X0GKQ4m7KkUuAe4GKgDlgsaWFEvJ5V7QZgZ0TMkLQAuAP4TNb2u4CnO1YiYg0wJ6v9TcCT2fUj4q/T6I+Z2UC3ZEPm+stHpvZPgklzBDMXqI2IdRHRDDwKzOtUZx7wULL8OHChknGbpCuBdcCqbtq/EHgrInKaNtrMbLCr2bCTirISZh87ql+Ol2aCmQRszFqvS8q6rBMRrcBuYKykYcDXgNsO0f4C4JFOZTdLWi7pQUn9k6LNzAaImg07OX3KaMrL+ufye5pH6eoKUuRY5zYyp7u6vFFbUjlwBfCjrOJ7gRPInELbAny7m31vlFQjqaa+vv7QPTAzKxL7m9tYtWk31f10egzSfaNlHTAla30ysLmbOnWSyoBRQANwNnCVpDuB0UC7pAMRcXey32XAaxGxraOh7GVJ9wP/3FVQEXEfcB9AdXV154RnZlaUltXtorU9+u36C6SbYBYDMyVNJ3MxfgHw2U51FgLXAy8DVwHPR0QA53dUkHQr0JSVXACuodPpMUkTI2JLsjofWNl3XTEzG9j6+wI/pJhgIqJV0s3AM0Ap8GBErJJ0O1ATEQuBB4CHJdWSGbks6KldSUeRuTPtC5023SlpDplTbOu72G5mNmjVrG9gxvjhjD6qvN+OmeYIhoh4CniqU9m3spYPAFf30Matndb3AWO7qHdtb2I1MytW7e3Ba+/s4rJTj+nX4/pJfjOzIrd2exO797f06+kxcIIxMyt6v67dAcDHZozr1+M6wZiZFbmXandw/LhhTBo9tF+P6wRjZlbEmlvbeWXdu5zbz6MXcIIxMytqSzfuYl9zG+fNdIIxM7M+9Ou19ZQIPnr8h26+TZ0TjJlZEXuxdgenTxnNqKFD+v3YTjBmZkVqz4EWlm3cxfl5uP4CTjBmZkXr5bfepT3IywV+cIIxMytav167g6PKSznjuPy8vcQJxsysSL1Uu4Ozp4/pt/e/dOYEY2ZWhDbt2s+6HXs5b2ZV3mJwgjEzK0Ir6nYBcNa0/L3c1wnGzKwIrd2WeSHwjPHD8xaDE4yZWRGqrW9i0uihHFWe6ltZDskJxsysCK3d1sTMCfkbvUDKCUbSpZLWSKqVdEsX2yskPZZsXyRpWqftx0lqkvTVrLL1klZIWiqpJqt8jKRnJa1NvvN34tHMLI/a2oO36puYUVWkCUZSKXAPcBkwC7hG0qxO1W4AdkbEDOAu4I5O2+8Cnu6i+QsiYk5EVGeV3QI8FxEzgeeSdTOzQWfTzv0cbG3P6/UXSHcEMxeojYh1EdEMPArM61RnHvBQsvw4cKEkAUi6ElgHrMrxeNltPQRc2YvYzcwGrNr6RoCiPkU2CdiYtV6XlHVZJyJagd3AWEnDgK8Bt3XRbgA/l7RE0o1Z5RMiYkvS1hZgfFdBSbpRUo2kmvr6+iPolplZYavdntxBVjUir3GkmWDURVnkWOc24K6IaOpi+7kRcSaZU29/IunjhxNURNwXEdURUV1Vlb8HkMzM0rJ2WxNVIyoYdVT/z6CcLc371+qAKVnrk4HN3dSpk1QGjAIagLOBqyTdCYwG2iUdiIi7I2IzQERsl/QkmVNxvwK2SZoYEVskTQS2p9g3M7OCVVsAF/gh3RHMYmCmpOmSyoEFwMJOdRYC1yfLVwHPR8b5ETEtIqYBfwP8j4i4W9IwSSMAktNolwAru2jreuAnaXXMzKxQRQS125ryfoEfUhzBRESrpJuBZ4BS4MGIWCXpdqAmIhYCDwAPS6olM3JZ0EOzE4Ank/sAyoAfRMS/JNv+F/BDSTcA7wBX93mnzMwK3PbGgzQebM37BX5I9xQZEfEU8FSnsm9lLR+gh0QQEbdmLa8DTu+m3rvAhb0I18xswHv/An/+E4yf5DczKyJrt2VuUZ5RACMYJxgzsyJSW9/EyMoyqoZX5DsUJxgzs2KSmYNsBMm16rxygjEzKyKFMAdZBycYM7MisXNvMzuamgviFmVwgjEzKxq19ckdZAVwgR+cYMzMisaarckklx7BmJlZX3pj6x5GVpYxafTQfIcCOMGYmRWN1VsaOXniyIK4gwycYMzMikJ7e7BmayOnHJPfKfqzOcGYmRWBup37aTrYyikTR+Y7lPc4wZiZFYHVW/cAcLITjJmZ9aU3tjQiwYkFcosyOMGYmRWF1Vv2MH3sMI4qT3WS/MPiBGNmVgTe2LqHkycWzgV+SDnBSLpU0hpJtZJu6WJ7haTHku2LJE3rtP04SU2SvpqsT5H0gqTVklZJ+lJW3VslbZK0NPlcnmbfzMwKxd6DrWxo2McpxxTO9RdIMcFIKgXuAS4DZgHXSJrVqdoNwM6ImAHcBdzRaftdwNNZ663AVyLiFOCjwJ90avOuiJiTfD7wojMzs2K1ZlsjEYV1gR/SHcHMBWojYl1ENAOPAvM61ZkHPJQsPw5cqOQJIUlXAuuAVR2VI2JLRLyWLDcCq4FJKfbBzKzgrd6SuYPslEF0imwSsDFrvY4PJ4P36kREK7AbGCtpGPA14LbuGk9Op50BLMoqvlnSckkPSjq6tx0wMxsI3tjSyIgCmiKmQ5oJpqu5CiLHOreROd3V1GXD0nDgx8CXI2JPUnwvcAIwB9gCfLubfW+UVCOppr6+vudemJkVuDe27uGUYwpnipgOaSaYOmBK1vpkYHN3dSSVAaOABuBs4E5J64EvA9+QdHNSbwiZ5PKPEfFER0MRsS0i2iKiHbifzCm6D4mI+yKiOiKqq6qqet9LM7M8igje2NJYcHeQAaR5w/RiYKak6cAmYAHw2U51FgLXAy8DVwHPR0QA53dUkHQr0BQRdyfXZx4AVkfEd7IbkjQxIrYkq/OBlX3fJTOzwlK3cz+NBTZFTIfUEkxEtCajjmeAUuDBiFgl6XagJiIWkkkWD0uqJTNyWdBDs+cC1wIrJC1Nyr6R3DF2p6Q5ZE6xrQe+0OedMjMrMB0X+E8uoEkuO6T6yGfyg/+pTmXfylo+AFzdQxu3Zi3/mq6v2xAR1/YmVjOzgWjlpt2UloiTCjDB+El+M7MBbFndbmaOH15QU8R0cIIxMxugIoLldbs4ffLofIfSJScYM7MBamPDfnbua+H0KU4wZmbWh5bV7QLgtMmj8hxJ15xgzMwGqOV1u6goKynIC/zgBGNmNmAt27ib2ceOZEhpYf4oL8yozMzskFrb2lmxaTenFegFfnCCMTMbkGrrm9jf0sacAr3AD04wZmYD0vKNu4HCvcAPTjBmZgPSsrpdjKgsY9rYYfkOpVtOMGZmA9Cy5AHLkpLCmqI/mxOMmdkAc6CljTe2NBb06TFwgjEzG3Be37KH1vYo6DvIwAnGzGzAWb4x8wT/6VM8gjEzsz60eMNOJo0eysRRQ/MdyiE5wZiZDSARwatvN3DWtKPzHUqPUk0wki6VtEZSraRbutheIemxZPsiSdM6bT9OUpOkr/bUpqTpSRtrkzbL0+ybmVk+bHh3H/WNBzlr+ph8h9Kj1BKMpFLgHuAyYBZwjaRZnardAOyMiBnAXcAdnbbfBTydY5t3AHdFxExgZ9K2mVlRefXtBgDOHswJBpgL1EbEuohoBh4F5nWqMw94KFl+HLhQkgAkXQmsA1b11GayzyeTNkjavDKFPpmZ5dWr6xsYM6ycE6qG5zuUHqWZYCYBG7PW65KyLutERCuwGxgraRjwNeC2HNscC+xK2ujuWGZmA97i9Q1UTz2a5HfxgpZmgumq95FjndvInO5qyrF+LsfKNCDdKKlGUk19fX1XVczMCtK2PQfY8O4+5g6A02MAZSm2XQdMyVqfDGzupk6dpDJgFNAAnA1cJelOYDTQLukAsKSbNncAoyWVJaOYro4FQETcB9wHUF1d3WUSMjMrRB3XX5xgYDEwU9J0YBOwAPhspzoLgeuBl4GrgOcjIoDzOypIuhVoioi7kyT0oTYjIiS9kLTxaNLmT1Lsm5lZv1u8voFh5aXMmjgy36HkJLVTZMlI4mbgGWA18MOIWCXpdklXJNUeIHPNpRb4M+BDtzLn0may+WvAnyVtjU3aNjMrGq++3cCZU4+mrEDfYNlZmiMYIuIp4KlOZd/KWj4AXN1DG7f21GZSvo7MXWZmZkVn974W1mxr5Pd/Z2K+Q8nZwEiDZmaDXM2GBiIYEA9YdnCCMTMbABa93cCQUhX0K5I7c4IxMxsAfvVmPdVTx1A5pDTfoeTMCcbMrMBt33OAN7Y28vETq/IdymHJOcFIOk/S55LlquRWYTMzS9mLa3cAcP7McXmO5PDklGAk/TmZ24C/nhQNAf4hraDMzOx9L66tZ9zw8gHz/EuHXEcw84ErgL0AEbEZGJFWUGZmltHeHry4dgfnzRhHSUnhzz+WLdcE05w8YR8AyWSUZmaWste37OHdvc2cP3NgXX+B3BPMDyX9PZn5vj4P/AK4P72wzMwMBu71F8jxSf6I+GtJFwN7gJOAb0XEs6lGZmZm/OrNek4+ZgTjR1bmO5TDllOCSU6JPR8Rz0o6CThJ0pCIaEk3PDOzwWtfcys1Gxr49+cOzJt2cz1F9iugQtIkMqfHPgd8P62gzMwMXln3Li1tMSCvv0DuCUYRsQ/4A+BvI2I+MCu9sMzM7Fdv7qBySAnV047OdyhHJOcEI+kc4N8BP0vKUp2J2cxsMIsInn19G+fNGDegpofJlmuC+RKZd7U8kbzTZTrwfHphmZkNbis37WHTrv1cMvuYfIdyxHIdhewD2oFrJP0hILp5572ZmfXeM6u2UiK46JQJ+Q7liOU6gvlH4EEy12D+DfCp5PuQJF0qaY2kWkkfelulpApJjyXbF0malpTPlbQ0+SyTND8pPymrfKmkPZK+nGy7VdKmrG2X59g3M7OC88yqrcydPoYxw8rzHcoRy3UEUx8RPz2chiWVAvcAFwN1wGJJCyPi9axqNwA7I2KGpAXAHcBngJVAdUS0SpoILJP004hYA8zJan8T8GRWe3dFxF8fTpxmZoVmXX0Ta7c38dmzB/a9VLkmmD+X9D3gOeBgR2FEPHGIfeYCtcmrjJH0KDAPyE4w84Bbk+XHgbslddyx1qGSrk/HXQi8FREbcuyDmdmA8MyqbQAD+voL5J5gPgecTGYW5fakLIBDJZhJwMas9Trg7O7qJKOV3cBYYIeks8mclpsKXBsRrZ32XQA80qnsZknXATXAVyJiZw59MzMrKM+s2srvTBrFpNFD8x1Kr+R6Deb0iKiOiOsj4nPJ59/3sE9X0352Hol0WyciFkXEbOAs4OuS3psnQVI5mdmdf5S1373ACWROoW0Bvt1lUNKNkmok1dTX1/fQBTOz/rV19wGWbtzF780euBf3O+SaYF6RdLgnA+uAKVnrk4HN3dWRVAaMAhqyK0TEajKvCTg1q/gy4LWI2JZVb1tEtEVEO5mJOOd2FVRE3Jcky+qqqoH5dKyZFa9nX98KwO8N8NNjkHuCOQ9YmtwRtlzSCknLe9hnMTBT0vRkxLEAWNipzkLg+mT5KjLznUWyTxmApKlkJthcn7XfNXQ6PZbcDNBhPpkbBczMBpSnVmzl+HHDmDF+eL5D6bVcr8FcergNJ9dUbgaeAUqBB5OHNG8HaiJiIfAA8LCkWjIjlwXJ7ucBt0hqIXPN54sRsQNA0lFk7kz7QqdD3ilpDplTbOu72G5mVtA279rPK2+/y5cunIk0sF4u1pVcp+s/oju1IuIp4KlOZd/KWj4AXN3Ffg8DD3fT5j4yNwJ0Lr/2SGI0MysU/7R0ExEw/4xJ+Q6lT+R6iszMzFIUETz52iY+MvVopo4tjpcGO8GYmRWAVZv3sHZ7U9GMXsAJxsysIDzx2ibKS0v41GkTe648QDjBmJnlWWtbOwuXbeaCk6sYfdTAnXusMycYM7M8e7F2BzuaDjL/jMn5DqVPOcGYmeXZk69tYtTQIVxwcnE9/O0EY2aWRzuaDvIvK7dy5ZxjqSgbmG+u7I4TjJlZHj22eCPNbe1ce87UfIfS55xgzMzypK09+MGid/jYCWOZMX5EvsPpc04wZmZ58tzqbWzatZ/rinD0Ak4wZmZ58/ArGzhmZCUXnTLwp+bvihOMmVkerKtv4sW1O/js2cdRVlqcP4qLs1dmZgXu4Vc2MKRULJg7pefKA5QTjJlZP9u9r4Uf1dRx2akTGT+isucdBignGDOzfvbQy+tpOtjKTZ84Id+hpMoJxsysH+092MqDL73NhSePZ9axI/MdTqpSTTCSLk1es1wr6ZYutldIeizZvkjStKR8rqSlyWeZpPlZ+6xPXtm8VFJNVvkYSc9KWpt8H51m38zMjsQPFr3Drn0t/MknZ+Q7lNSllmAklQL3AJcBs4BrJM3qVO0GYGdEzADuAu5IylcC1RExh8zrmv9eUvbbNy+IiDkRUZ1VdgvwXETMBJ5L1s3MCsaBljbuf3EdHzthLGceV/y/A6c5gpkL1EbEuohoBh4F5nWqMw94KFl+HLhQkiJiX0S0JuWVQORwvOy2HgKu7FX0ZmZ97PEldWxvPMjNFxT/6AXSTTCTgI1Z63VJWZd1koSyGxgLIOlsSauAFcBNWQkngJ9LWiLpxqy2JkTElqStLcD4roKSdKOkGkk19fX1veqgmVmuDra2ce8v32LOlNGcc8LYfIfTL9JMMOqirPNIpNs6EbEoImYDZwFfl9RxL9+5EXEmmVNvfyLp44cTVETcFxHVEVFdVVVcU2ObWeF6+OUNbNq1n69cciJSVz/6ik+aCaYOyH6CaDKwubs6yTWWUUBDdoWIWA3sBU5N1jcn39uBJ8mcigPYJmli0tZEYHsf9sXM7Ijt3tfC3z5fy8dPrOL8mYPnF9s0E8xiYKak6ZLKgQXAwk51FgLXJ8tXAc9HRCT7lAFImgqcBKyXNEzSiKR8GHAJmRsCOrd1PfCTlPplZnZY7n5hLXsOtPD1y07Odyj9qqznKkcmIlol3Qw8A5QCD0bEKkm3AzURsRB4AHhYUi2ZkcuCZPfzgFsktQDtwBcjYoek44Enk+FlGfCDiPiXZJ//BfxQ0g3AO8DVafXNzCxXGxv28dBvNnDVmZM5ZWJxP/fSmSJyuUGrOFVXV0dNTU3PFc3MjtCfPvJbfv76Vn751Qs4ZlRxTAsjaUmnx0S65Cf5zcxS8pu3drBw2WY+f/7xRZNcDocTjJlZCg60tPHNJ1dy3Jij+OLvDo7nXjpL7RqMmdlgds8Ltby9Yy8P3zCXoeWl+Q4nLzyCMTPrY2u2NnLvL9/iD86YNKhuS+7MCcbMrA+1tQffeHIFIyrL+Obvn5LvcPLKCcbMrA/d+8talmzYyX/71CzGDq/Idzh55QRjZtZHlmxo4K5frOWK049l/hmdp14cfJxgzMz6wO79LfzpI0s5dnQlfzH/1EEz39ih+C4yM7Neishcd9m25wA/uukcRlYOyXdIBcEjGDOzXvrei2/zs+Vb+LNLTuSMQfAisVw5wZiZ9cJzq7fxP55ezeW/cww3ffyEfIdTUJxgzMyO0JqtjfzpI7/l1GNH8e2r51BS4usu2ZxgzMyOwPbGA9zw0GKGVZRx/3XVg/Zp/UPxRX4zs8PUsLeZP/zeIhr2NvPI5z86KCeyzIVHMGZmh2H3/haufWARG97dx/euq+b0KaPzHVLBSjXBSLpU0hpJtZJu6WJ7haTHku2LJE1LyudKWpp8lkman5RPkfSCpNWSVkn6UlZbt0ralLXf5Wn2zcwGn8YDLXzu/77Km9sa+e61H+FjM8blO6SCltopMkmlwD3AxUAdsFjSwoh4PavaDcDOiJghaQFwB/AZMq9Brk7eijkRWCbpp0Ar8JWIeC15dfISSc9mtXlXRPx1Wn0ys8GrvvEgn/v+q6ze0sg9nz2DC04an++QCl6aI5i5QG1ErIuIZuBRYF6nOvOAh5Llx4ELJSki9kVEa1JeCQRARGyJiNeS5UZgNeD5GMwsVe+8u4+rvvsbarc38b3rqrn01In5DmlASDPBTAI2Zq3X8eFk8F6dJKHsBsYCSDpb0ipgBXBTVsIh2T4NOANYlFV8s6Tlkh6U5KedzKzXlm3cxae/+xt272/hB5//KBec7JFLrtJMMF3dEB651omIRRExGzgL+Lqk927TkDQc+DHw5YjYkxTfC5wAzAG2AN/uMijpRkk1kmrq6+sPpz9mNsj8cPFGrv77lykvLeHxm87hTD+lf1jSTDB1wJSs9cnA5u7qSCoDRgEN2RUiYjWwFzg1qTeETHL5x4h4Iqvetohoi4h24H4yp+g+JCLui4jqiKiuqhq8LwIys+4dbG3jG0+u4L/8eDlzp43hp//xPGaMH5HvsAacNBPMYmCmpOmSyoEFwMJOdRYC1yfLVwHPR0Qk+5QBSJoKnASsV2Z60geA1RHxneyGkpsBOswnc6OAmdlhWb1lD/PufokfLHqHmz5xAt//3FmMGVae77AGpNTuIkvuALsZeAYoBR6MiFWSbgdqImIhmWTxsKRaMiOXBcnu5wG3SGoB2oEvRsQOSecB1wIrJC1N6n4jIp4C7pQ0h8wptvXAF9Lqm5kVn7b24P4X1/Gdn7/JyKFlfO+6ai6aNSHfYQ1oiuh8WWTwqK6ujpqamnyHYWZ5tmzjLv7bT1ayvG43l84+hr+cf+qgfxvloUhaEhHVPdXzVDFmNmg17G3mr55Zw6OL36FqeAX/e8Ecrjj9WL8srI84wZjZoNN4oIXvvfg2D/z6bfa3tHHDudP50kUzGeEXhfUpJxgzGzR272vhHxZt4P4X17FrXwuXnXoMX7nkRN8hlhInGDMrehsb9vF/X1rPo4vfYV9zGx8/sYqvXnIip032RJVpcoIxs6LU0tbOL17fxiOLN/Li2npKJa44/Vj++PzjmXXsyHyHNyg4wZhZ0WhvDxavb2Dhss08vXIrDXubmTiqkj/95EwWzJ3CxFFD8x3ioOIEY2YD2v7mNl6q3cFzb2zjudXb2d54kKFDSrlo1gTmn3EsnzhxPKV+lXFeOMGY2YDS3NrOys27efmtd3mpdgc1G3bS3NrO8IoyPnFiFZfMnsDFsyZwVLl/vOWb/wbMrGBFBHU797Ny026Wb9rNkg07WbZxFwdb2wE4+ZgRXPvRqfzuSVWcPX0s5WV+SW8hcYIxs7yLCOqbDrKufi+125t4c1sja7Y2smZbI7v2tQBQViJmTxrFH350Kh+ZejRzp49hnJ+2L2hOMGaWuvb2YOe+ZrbuOcCWXQfYvHs/m3bu552GfZnPu/toPPj+K5+GV5Rx4oThXHbqRGYfO5LTJo/ipGNGUFFWmsde2OFygjGzwxIR7Gtuo+lgK7v3t2Q++1rYtb+FnXubadjXzLtNB3m3qZkdTQfZ0dTM9sYDtLR9cN7D8tISJo8ZynFjjuIjU4/m+HHDOL5qOMdXDWPS6KGerqUIOMGYDTDt7UFbBG3tQWt70NYWtLS309YetLS109oWtLa309ya+W5pyyw3t7XT3Jp82to42NLOwdZ2DrS0caClnQOtbclyG/uaM5/9zW3sbW5l78FW9h7MJJWmg620tXc/Se6QUnH0UeWMG17BuBEVnDB+OBNGVjJhRAUTRlZy7OihTBxdybhhFZT47q6i5gTTj9rag9+8tYOnV26l6UBrzzvkoK/mwj7cWbUPWTuHpiKrUneHzi7vrn50U58u6gfv9zOytr2//MF9Oo4Z8f76e/U7rUdE8v3+cntktrd3lEWmrD2rblt7ZNWL99Yz3x9MJu2RSShpTYBeViKGDimlsryUoUNKOao88xlWXsaEEZUMqyhjeEUpwyvLGFE5hBGVZYysHMKooUMYfVTme8ywcoZXlHn0YYATTL/Y39zG3S+s5fEldWzbc5ARFWWMG9F3Fyf77L/yYTZ0qOq5/IDJrtFddWXV6rZO1obu2uxYFspafn+jsuq9v9y5PLNRgEpAlGTK9X67HfuUZC+XZL4lKC3Re+2USpQISiRKSj64XCpl6ib1Sksy5WUlmeVSibLSEsqS8iGlmfIhJSWUlWa2DSkR5WUlDCnNfMrLSihPvivKSqgYklmvHFJKRVkJZaW+A8v6lhNMyrY3HuDz/28Jy+t28cmTxvOtT03mwlPGUznEFyvNrLil+iuLpEslrZFUK+mWLrZXSHos2b5I0rSkfK6kpclnmaT5PbWZvGZ5kaS1SZt5f8fpmq2NzL/nN7y5tZHv/uFHeOCPzuL3T5vo5GJmg0JqCUZSKXAPcBkwC7hG0qxO1W4AdkbEDOAu4I6kfCVQHRFzgEuBv5dU1kObdwB3RcRMYGfSdt6s2drIp+/9Da3t7fzopnP4vdnH5DMcM7N+l+YIZi5QGxHrIqIZeBSY16nOPOChZPlx4EJJioh9EdFxFbyS96/DdtmmMifhP5m0QdLmlan0Kgetbe3858eXUVFWwpNfPJdTJ43KVyi8v02EAAALH0lEQVRmZnmTZoKZBGzMWq9LyrqskySU3cBYAElnS1oFrABuSrZ31+ZYYFdWUurqWP3mgV+/zfK63dx6xWyOHe3ZW81scEozwXR1z0/nGyy7rRMRiyJiNnAW8HVJlYeon8uxMgeUbpRUI6mmvr6+2+CP1Lr6Jr7z7JtcPGsCnzptYp+3b2Y2UKSZYOqAKVnrk4HN3dWRVAaMAhqyK0TEamAvcOoh2twBjE7a6O5YHe3dFxHVEVFdVVV1BN3qXnt7cMuPV1BRVsJfXHmqnwUws0EtzQSzGJiZ3N1VDiwAFnaqsxC4Plm+Cng+IiLZpwxA0lTgJGB9d21G5gm5F5I2SNr8SXpd69rCZZt5dX0D//VTs5gwsrK/D29mVlBSew4mIlol3Qw8A5QCD0bEKkm3AzURsRB4AHhYUi2ZkcuCZPfzgFsktQDtwBcjYgdAV20m+3wNeFTSXwC/TdruNxHBd//1LU6cMJyrzpzcn4c2MytIOtwpQopJdXV11NTU9ElbL7yxnc99fzHfvvp0Pv0RJxgzK16SlkREdU/1PDdEH7n3l29x7KhKrphzbL5DMTMrCE4wfWDJhp28ur6BPz7/eIZ4PiczM8AJpk9891/fYvRRQ1gwd0rPlc3MBgknmF6q3d7Is69v47pzpnFUuecONTPr4ATTSw+/vIHyshKuP2dqvkMxMysoTjC9sL+5jSd+u4nLTj2GscP77v0uZmbFwAmmF55asYXGA61cM/e4fIdiZlZwnGB64ZFX3+H4ccM4e/qYfIdiZlZwnGCO0NptjdRs2MmCuVM855iZWRecYI7QI69uZEip+LSnhTEz65ITzBE40NLGE7+t45LZvrhvZtYdJ5gj8Myqreza18I1Z/nivplZd5xgjsCw8jIunjWBj50wNt+hmJkVLD96fgQumjWBi2ZNyHcYZmYFzSMYMzNLhROMmZmlItUEI+lSSWsk1Uq6pYvtFZIeS7YvkjQtKb9Y0hJJK5LvTyblIyQtzfrskPQ3ybY/klSfte2P0+ybmZkdWmrXYCSVAvcAFwN1wGJJCyPi9axqNwA7I2KGpAXAHcBngB3Av4mIzZJOJfOK5EkR0QjMyTrGEuCJrPYei4ib0+qTmZnlLs0RzFygNiLWRUQz8Cgwr1OdecBDyfLjwIWSFBG/jYjNSfkqoFLSBx44kTQTGA+8mFoPzMzsiKWZYCYBG7PW65KyLutERCuwG+h87++ngd9GxMFO5deQGbFEdl1JyyU9Lslv/zIzy6M0E0xXE3TF4dSRNJvMabMvdFFvAfBI1vpPgWkRcRrwC94fGX3wgNKNkmok1dTX1x8ifDMz6400E0wdkD2KmAxs7q6OpDJgFNCQrE8GngSui4i3sneSdDpQFhFLOsoi4t2sUc79wEe6Cioi7ouI6oiorqqqOtK+mZlZD9J80HIxMFPSdGATmRHHZzvVWQhcD7wMXAU8HxEhaTTwM+DrEfFSF21fwwdHL0iaGBFbktUrgNU9BbhkyZIdkjYcRp+6M47MjQnFwH0pTMXSl2LpBwzuvuT0Cl998BJG35J0OfA3QCnwYET8paTbgZqIWCipEngYOIPMyGVBRKyT9F+BrwNrs5q7JCK2J+2uAy6PiDeyjvU/ySSW1qSt/5C9PU2SaiKiuj+OlTb3pTAVS1+KpR/gvuTUbpoJZrDwP7TC5L4UnmLpB7gvufCT/GZmlgonmL5xX74D6EPuS2Eqlr4USz/AfemRT5GZmVkqPIIxM7NUOMEcAUlXS1olqV1StxfGJK1PJuxcKqmmP2PM1WH05ZATlxYCSWMkPStpbfJ9dDf12rImRV3Y33F250gnhy1EOfRlQExOK+lBSdslrexmuyT9n6SfyyWd2d8x5iqHvvyupN1Zfyff6vVBI8Kfw/wApwAnAb8Eqg9Rbz0wLt/x9rYvZG4zfws4HigHlgGz8h17F3HeCdySLN8C3NFNvaZ8x3okf8bAF4HvJssLyEyVlPfYj7AvfwTcne9Yc+jLx4EzgZXdbL8ceJrMrCQfBRblO+Ze9OV3gX/uy2N6BHMEImJ1RKzJdxx9Ice+5DJxaSHInjz1IeDKPMZyuI54cth+jDFXA+XfS48i4lcks4t0Yx7w/yLjFWC0pIn9E93hyaEvfc4JJl0B/Dx5p82N+Q6mF3KZuLQQTIhkNofke3w39SqT+ehekVQoSaivJoctBLn+eymGyWkHyv+NXJ0jaZmkp5O5IHslzaliBjRJvwCO6WLTNyPiJzk2c25k3mkzHnhW0hvJbxH9qg/6ksvEpf3iUH05jGaOS/5ejgeel7QiOs13lwe9nhy2gOQS50+BRyLioKSbyIzMPpl6ZH1voPyd5OI1YGpENCWzsPwTMLM3DTrBdCMiLuqDNjYn39slPUnm1EG/J5g+6EsuE5f2i0P1RdK2jjnpktMU27tpo+PvZZ2kX5KZqijfCeZwJoet6zw5bIHpsS8R8W7W6v1kZk0fiArm/0ZvRcSerOWnJP2dpHERccTzrfkUWUokDZM0omMZuATo8u6NAeC9iUsllZO5wFwwd19l6Zg8leT7Q6MzSUcreXmdpHHAucDrnevlQS5/xtn9e29y2H6MMVc99qXTdYqcJqctUAuB65K7yT4K7I73J90dUCQd03FNT9JcMvnh3UPv1YN839kwED/AfDK/uRwEtgHPJOXHAk8ly8eTuXtmGZm3cn4z33EfaV+S9cuBN8n8pl+ofRkLPEdmktTngDFJeTXwvWT5Y8CK5O9lBXBDvuM+1J8xcDtwRbJcCfwIqAVeBY7Pd8y96Mv/TP5fLANeAE7Od8zd9OMRYAvQkvw/uQG4Cbgp2S4yr4Z/K/n31O1dpfn+5NCXm7P+Tl4BPtbbY/pJfjMzS4VPkZmZWSqcYMzMLBVOMGZmlgonGDMzS4UTjJmZpcIJxuwISGrq5f6PJzMJHKrOLw81w3WudTrVr5L0L7nWN+sNJxizfpbM8VQaEev6+9gRUQ9skXRufx/bBh8nGLNeSJ7g/itJK5V5989nkvKSZKqNVZL+WdJTkq5Kdvt3ZM0yIOneZPLNVZJu6+Y4TZK+Lek1Sc9JqsrafLWkVyW9Ken8pP40SS8m9V+T9LGs+v+UxGCWKicYs975A2AOcDpwEfBXyTQofwBMA34H+GPgnKx9zgWWZK1/MyKqgdOAT0g6rYvjDANei4gzgX8F/jxrW1lEzAW+nFW+Hbg4qf8Z4P9k1a8Bzj/8rpodHk92adY755GZFbgN2CbpX4GzkvIfRUQ7sFXSC1n7TATqs9b/bfI6h7Jk2yxgeafjtAOPJcv/ADyRta1jeQmZpAYwBLhb0hygDTgxq/52MlMBmaXKCcasd7p74dehXgS2n8y8YkiaDnwVOCsidkr6fse2HmTP8XQw+W7j/f/T/4nM3HKnkzlTcSCrfmUSg1mqfIrMrHd+BXxGUmlyXeTjZCai/DWZF2qVSJpA5nW0HVYDM5LlkcBeYHdS77JujlNCZgZlgM8m7R/KKGBLMoK6lsxrjDucyMCd2dsGEI9gzHrnSTLXV5aRGVX8l4jYKunHwIVkfpC/CSwi8wZKgJ+RSTi/iIhlkn5LZhbbdcBL3RxnLzBb0pKknc/0ENffAT+WdDWZ2Yr3Zm27IInBLFWeTdksJZKGR+btgGPJjGrOTZLPUDI/9M9Nrt3k0lZTRAzvo7h+BcyLiJ190Z5ZdzyCMUvPP0saDZQD/z0itgJExH5Jf07m3e3v9GdAyWm87zi5WH/wCMbMzFLhi/xmZpYKJxgzM0uFE4yZmaXCCcbMzFLhBGNmZqlwgjEzs1T8f9THFEtoxGPoAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "alpha is: 0.029506416260636846\n"
     ]
    }
   ],
   "source": [
    "mses = np.mean(lasso.mse_path_, axis = 1)\n",
    "plt.plot(np.log10(lasso.alphas_), mses) \n",
    "#plt.plot(np.log10(lasso.alphas_)*np.ones(3), [0.3, 0.4, 1.0])\n",
    "plt.xlabel('log(alpha)')\n",
    "plt.ylabel('mse')\n",
    "plt.show()    \n",
    "#打印叉验证最佳alpha值：            \n",
    "print ('alpha is:', lasso.alpha_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>columns</th>\n",
       "      <th>coef_lr</th>\n",
       "      <th>coef_ridge</th>\n",
       "      <th>coef_lasso</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>33</th>\n",
       "      <td>yr</td>\n",
       "      <td>0.523551</td>\n",
       "      <td>0.173104</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>temp</td>\n",
       "      <td>0.305429</td>\n",
       "      <td>0.204613</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>mnth_9</td>\n",
       "      <td>0.148181</td>\n",
       "      <td>0.078043</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>atemp</td>\n",
       "      <td>0.114507</td>\n",
       "      <td>0.177908</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>mnth_10</td>\n",
       "      <td>0.106982</td>\n",
       "      <td>0.009764</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>weathersit_1</td>\n",
       "      <td>0.105201</td>\n",
       "      <td>0.105251</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>season_4</td>\n",
       "      <td>0.095557</td>\n",
       "      <td>0.088266</td>\n",
       "      <td>-0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>mnth_12</td>\n",
       "      <td>0.067452</td>\n",
       "      <td>-0.094907</td>\n",
       "      <td>-0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>mnth_8</td>\n",
       "      <td>0.059572</td>\n",
       "      <td>0.023664</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>weathersit_2</td>\n",
       "      <td>0.047123</td>\n",
       "      <td>0.044738</td>\n",
       "      <td>-0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>mnth_11</td>\n",
       "      <td>0.046841</td>\n",
       "      <td>-0.083505</td>\n",
       "      <td>-0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>weekday_6</td>\n",
       "      <td>0.027430</td>\n",
       "      <td>0.026175</td>\n",
       "      <td>-0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>workingday</td>\n",
       "      <td>0.024960</td>\n",
       "      <td>0.024455</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>mnth_6</td>\n",
       "      <td>0.021836</td>\n",
       "      <td>0.042529</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>season_2</td>\n",
       "      <td>0.009795</td>\n",
       "      <td>0.012770</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>weekday_5</td>\n",
       "      <td>0.008918</td>\n",
       "      <td>0.009365</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>weekday_3</td>\n",
       "      <td>0.007647</td>\n",
       "      <td>0.006887</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>weekday_4</td>\n",
       "      <td>0.005022</td>\n",
       "      <td>0.007225</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>mnth_5</td>\n",
       "      <td>0.000576</td>\n",
       "      <td>0.044606</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>instant</td>\n",
       "      <td>-0.000796</td>\n",
       "      <td>0.000163</td>\n",
       "      <td>0.000628</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>weekday_2</td>\n",
       "      <td>-0.003675</td>\n",
       "      <td>-0.003928</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>season_3</td>\n",
       "      <td>-0.015049</td>\n",
       "      <td>-0.010514</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>weekday_0</td>\n",
       "      <td>-0.022045</td>\n",
       "      <td>-0.022072</td>\n",
       "      <td>-0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>weekday_1</td>\n",
       "      <td>-0.023297</td>\n",
       "      <td>-0.023651</td>\n",
       "      <td>-0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>mnth_7</td>\n",
       "      <td>-0.023371</td>\n",
       "      <td>-0.027905</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>holiday</td>\n",
       "      <td>-0.030345</td>\n",
       "      <td>-0.028558</td>\n",
       "      <td>-0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>mnth_4</td>\n",
       "      <td>-0.055881</td>\n",
       "      <td>0.012230</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>mnth_3</td>\n",
       "      <td>-0.062292</td>\n",
       "      <td>0.034348</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>season_1</td>\n",
       "      <td>-0.090303</td>\n",
       "      <td>-0.090522</td>\n",
       "      <td>-0.039253</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>windspeed</td>\n",
       "      <td>-0.135164</td>\n",
       "      <td>-0.125146</td>\n",
       "      <td>-0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>mnth_2</td>\n",
       "      <td>-0.138876</td>\n",
       "      <td>-0.016466</td>\n",
       "      <td>-0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>hum</td>\n",
       "      <td>-0.149401</td>\n",
       "      <td>-0.131431</td>\n",
       "      <td>-0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>weathersit_3</td>\n",
       "      <td>-0.152324</td>\n",
       "      <td>-0.149989</td>\n",
       "      <td>-0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>mnth_1</td>\n",
       "      <td>-0.171022</td>\n",
       "      <td>-0.022402</td>\n",
       "      <td>-0.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         columns   coef_lr  coef_ridge  coef_lasso\n",
       "33            yr  0.523551    0.173104    0.000000\n",
       "27          temp  0.305429    0.204613    0.000000\n",
       "13        mnth_9  0.148181    0.078043    0.000000\n",
       "28         atemp  0.114507    0.177908    0.000000\n",
       "14       mnth_10  0.106982    0.009764    0.000000\n",
       "17  weathersit_1  0.105201    0.105251    0.000000\n",
       "4       season_4  0.095557    0.088266   -0.000000\n",
       "16       mnth_12  0.067452   -0.094907   -0.000000\n",
       "12        mnth_8  0.059572    0.023664    0.000000\n",
       "18  weathersit_2  0.047123    0.044738   -0.000000\n",
       "15       mnth_11  0.046841   -0.083505   -0.000000\n",
       "26     weekday_6  0.027430    0.026175   -0.000000\n",
       "32    workingday  0.024960    0.024455    0.000000\n",
       "10        mnth_6  0.021836    0.042529    0.000000\n",
       "2       season_2  0.009795    0.012770    0.000000\n",
       "25     weekday_5  0.008918    0.009365    0.000000\n",
       "23     weekday_3  0.007647    0.006887    0.000000\n",
       "24     weekday_4  0.005022    0.007225    0.000000\n",
       "9         mnth_5  0.000576    0.044606    0.000000\n",
       "0        instant -0.000796    0.000163    0.000628\n",
       "22     weekday_2 -0.003675   -0.003928    0.000000\n",
       "3       season_3 -0.015049   -0.010514    0.000000\n",
       "20     weekday_0 -0.022045   -0.022072   -0.000000\n",
       "21     weekday_1 -0.023297   -0.023651   -0.000000\n",
       "11        mnth_7 -0.023371   -0.027905    0.000000\n",
       "31       holiday -0.030345   -0.028558   -0.000000\n",
       "8         mnth_4 -0.055881    0.012230    0.000000\n",
       "7         mnth_3 -0.062292    0.034348    0.000000\n",
       "1       season_1 -0.090303   -0.090522   -0.039253\n",
       "30     windspeed -0.135164   -0.125146   -0.000000\n",
       "6         mnth_2 -0.138876   -0.016466   -0.000000\n",
       "29           hum -0.149401   -0.131431   -0.000000\n",
       "19  weathersit_3 -0.152324   -0.149989   -0.000000\n",
       "5         mnth_1 -0.171022   -0.022402   -0.000000"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 看看各特征的权重系数，系数的绝对值大小可视为该特征的重要性\n",
    "fs = pd.DataFrame({\"columns\":list(feat_names), \"coef_lr\":list((lr.coef_.T)), \"coef_ridge\":list((ridge.coef_.T)), \"coef_lasso\":list((lasso.coef_.T))})\n",
    "fs.sort_values(by=['coef_lr'],ascending=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The RMSE of LinearRegression on test is 0.09370397005135352\n",
      "The RMSE of LinearRegression on train is 0.0854526405057376\n",
      "The RMSE of ridgeRegression on test is 0.09342398394396466\n",
      "The RMSE of ridgerRegression on train is 0.08599183308329299\n",
      "The RMSE of lassoRegression on test is 0.1643648008616637\n",
      "The RMSE of lassoRegression on train is 0.16561621048101285\n"
     ]
    }
   ],
   "source": [
    "import math\n",
    "mset1 = ( mean_squared_error(y_test, y_test_pred_lr))\n",
    "print ('The RMSE of LinearRegression on test is',math.sqrt(mset1))\n",
    "#训练集\n",
    "msea1= (mean_squared_error(y_train, y_train_pred_lr))\n",
    "print ('The RMSE of LinearRegression on train is',math.sqrt(msea1))\n",
    "mset2 = ( mean_squared_error(y_test, y_test_pred_ridge))\n",
    "print ('The RMSE of ridgeRegression on test is',math.sqrt(mset2))\n",
    "#训练集\n",
    "msea2= (mean_squared_error(y_train, y_train_pred_ridge))\n",
    "print ('The RMSE of ridgerRegression on train is',math.sqrt(msea2))\n",
    "mset3 = ( mean_squared_error(y_test, y_test_pred_lasso))\n",
    "print ('The RMSE of lassoRegression on test is',math.sqrt(mset3))\n",
    "#训练集\n",
    "msea3= (mean_squared_error(y_train, y_train_pred_lasso))\n",
    "print ('The RMSE of lassoRegression on train is',math.sqrt(msea3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 4、结论：通过三个模型的训练和测试我们可以发现：在对预测值cnt进行标准化后，每个模型的特征系数的绝对值小于1，我们横向比较，在加入正则化后的岭回归和lasso回归，特征系数小于最小二乘回归的系数，同时lasso回归对权重系数的收缩更加明显。接下来我们看看模型的评价指标RMSE，通过对比我们发现：三个模型都没有过拟合现象，在训练集和测试集上的表现都比较良好，但是纵向对比我们看到lasso回归的表现欠佳，rmse的值相对比较大，比其他两个模型的拟合效果稍欠佳，综上，通过模型评价指标rmse和回归系数权重来看，个人认为在测试集上表现最好的是岭回归模型。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
